Hyperparameter

noun

say it: “HY-per-puh-ram-ih-ter” /ˈhaɪpərpəˌræmətər/

In machine learning, a setting chosen before training that controls how a model learns, such as a tree's maximum depth.

What is a hyperparameter?

A hyperparameter is a setting you choose before training, such as a tree's max_depth or the k in k-nearest neighbors. Unlike a coefficient, the model doesn't learn it from the data; you pick it by comparing scores on a validation set or with cross-validation.

from sklearn.tree import DecisionTreeClassifier
tree = DecisionTreeClassifier(max_depth=3)   # a hyperparameter, set before fit()
print(tree.max_depth)   # 3

In a sentence

  • “The tree's depth is a hyperparameter: you choose it, the model doesn't learn it.”
  • “Grid search tried every combination of two values for each hyperparameter.”

Level: Journeyman · More AI Basics terms at this level

Updated