Regularization
noun
say it: “reg-yuh-ler-ih-ZAY-shun” /ˌrɛɡjələrəˈzeɪʃən/
In machine learning, a penalty on a model's coefficients while it learns, which keeps them small so the model is less likely to overfit.
What is regularization?
Regularization adds a penalty for large coefficients while a model learns, so it can't lean too hard on any one
feature, which helps against overfitting. In scikit-learn's LogisticRegression, C sets it: a smaller C means a
stronger penalty and smaller coefficients.
from sklearn.linear_model import LogisticRegression
links = [[0], [1], [2], [6], [8], [9]] # links in each email
spam = [0, 0, 0, 1, 1, 1]
for c in (100, 0.1):
print(LogisticRegression(C=c).fit(links, spam).coef_.round(2)) # [[2.55]], then [[0.39]]
In a sentence
- “Stronger regularization shrank every coefficient toward zero.”
- “Lowering C turned up the regularization on the spam model.”
Level: Freelancer · More AI Basics terms at this level
Updated