Bias and Fairness
A model learns the patterns in its data, including unfair ones. See where bias comes from, measure a model's results group by group (how often each group is selected, and how often each is wrongly turned away), and face the hard part honestly: some fairness goals can't all be met at once, and removing a sensitive column doesn't remove the bias. The running example is a scholarship fund's interview screen.
Updated
Where Bias Comes From
- Where Bias Comes From (Journeyman): A model trained on past decisions learns the old decision-makers' habits along with their judgment, and bias also gets in through groups missing from the data, groups measured less well, proxies and feedback loops, so leaving the sensitive column out of the features doesn't take the gap out of the model.
Measuring Fairness
- Measuring Results Group by Group (Freelancer): One score for everyone hides how a model treats each group, so you split the test rows by group, count each group's own confusion matrix, and compare three rates, the selection rate, the true positive rate and the false positive rate, whose gaps are what Fairlearn's demographic parity and equalized odds ask to be zero.
- Fairness Trade-offs (Freelancer): When two groups have different base rates, a useful screen can't give them equal selection rates and equal error rates at once, and one whose invitations mean the same in both groups can't have equal error rates either unless it makes no mistakes, as Barocas, Hardt and Narayanan, Chouldechova, and Kleinberg, Mullainathan and Raghavan show, so which goal matters is a choice for people, and sometimes the answer is no model.
Exercise: Fairness Audit
- Exercise: Fairness Audit (Freelancer): Write two functions that audit the Summit Scholarship Fund's interview screen, one that works out each district's selection rate, true positive rate and false positive rate from its own counts, reporting a rate that would divide by 0 as None, and one that finds the largest gap between districts, and let pytest check the exact rates.