AI Fairness Metrics: Demographic Parity to Equalized Odds Quiz

Answers and explanations appear as you take the quiz.

1. In the guide's loan example, group A has a 40% selection rate and group B has a 20% selection rate. Which metric does this violate?
   • Equal opportunity
   • Demographic parity
   • Calibration
   • Individual fairness
2. Equal opportunity requires which quantity to be equal across groups?
   • False positive rate
   • Selection rate
   • True positive rate
   • Positive predictive value
3. What does equalized odds add on top of equal opportunity?
   • Equal false positive rates
   • Equal selection rates
   • Equal group sizes
   • Equal base rates
4. In the example, group B's false positive rate is 6 of 80. Why is the denominator 80?
   • It is the number of approved applicants
   • It is the total population of both groups
   • It is 100 minus the 20 approved applicants
   • It is the number of group B applicants who would not repay
5. What do the impossibility results from Chouldechova and Kleinberg et al. say?
   • No model can be accurate for more than one group
   • When base rates differ, an imperfect classifier cannot satisfy predictive parity or calibration together with equal false positive and false negative rates
   • Demographic parity always implies equalized odds
   • Fairness metrics apply only to linear models
6. Why does removing the protected attribute from the inputs not guarantee fairness?
   • Models always add the attribute back automatically
   • Laws require the attribute to be included
   • Correlated proxies such as zip code can carry the same information
   • Removing features always lowers accuracy to zero
7. Which technique chooses group-specific thresholds, sometimes randomized, to equalize true and false positive rates?
   • Hardt et al.'s post-processing, available as Fairlearn's ThresholdOptimizer
   • AIF360 Reweighing
   • Dropout regularization
   • Principal component analysis
8. Why should fairness metrics be reported with confidence intervals and for intersectional subgroups?
   • Because the four-fifths rule requires exactly three decimal places
   • Because intersectional groups always have identical rates
   • Because Fairlearn cannot calculate point estimates
   • Because small groups make rate estimates noisy, and a model can look fair on single attributes while failing a combination of them