Calibration
noun
say it: “kal-uh-BRAY-shun” /ˌkæləˈbreɪʃən/
How closely the probabilities a model gives match how often things really happen.
What is calibration for a model's probabilities?
Calibration is how well a model's probabilities match what really happens: of all the cases it gives 0.7, about 7 in 10 should turn out positive. A well-calibrated model's 0.7 means the same in every group, a fairness goal that, when base rates differ, can't be met together with equalized odds.
In a sentence
- “The audit checked calibration separately for each district.”
- “Good calibration meant the model's 0.7 really did come true about 70% of the time.”
Level: Freelancer · More AI Basics terms at this level
Updated