Validation Set

noun

say it: “val-ih-DAY-shun SET” /ˌvæləˈdeɪʃən ˈsɛt/

A part of the data held back from training and used to compare models or choose settings, separate from the final test data.

What is a validation set?

A validation set is rows held back to compare models and choose settings, such as a tree's depth. Because you keep whatever does best on it, its score ends up a little flattering, so a separate test set is saved for one final, honest score. Cross-validation can stand in for it.

from sklearn.model_selection import train_test_split
rows = list(range(100))
rest, test = train_test_split(rows, test_size=0.2, random_state=0)
train, valid = train_test_split(rest, test_size=0.25, random_state=0)
print(len(train), len(valid), len(test))   # 60 20 20

In a sentence

  • “The test set stayed untouched until the validation set had picked the model.”
  • “She tried depths 2 to 10 on the validation set and kept 4.”

Level: Journeyman · More AI Basics terms at this level

Updated