COMPAS and Bias in Recidivism Algorithms Quiz

Answers and explanations appear as you take the quiz.

1. Among defendants who did NOT reoffend, what did ProPublica find about Black defendants compared with white defendants?
   • They were less likely to be scored at all
   • They had identical risk labels
   • They were nearly twice as likely to be labeled higher risk
   • They were more likely to be labeled lower risk
2. What was the core of Northpointe's defense of COMPAS?
   • At each score level, Black and white defendants reoffended at similar rates (calibration)
   • The tool used race to correct for historical bias
   • ProPublica had analyzed the wrong county
   • The tool was never used in any court
3. Under what condition does the impossibility result say calibration and equal error rates cannot all hold?
   • When the model uses more than 100 features
   • When the dataset has fewer than 10,000 people
   • When race is included as an input
   • When groups have different base rates and the predictor is not perfect
4. What did the Wisconsin Supreme Court hold in State v. Loomis?
   • COMPAS is unconstitutional and banned from courts
   • Judges may consider COMPAS, but it cannot be the deciding factor, and written cautions are required
   • COMPAS scores must decide the length of sentences
   • Defendants must be shown the full source code
5. Is race an input to COMPAS?
   • Yes, it is the most heavily weighted factor
   • Yes, but only for defendants in Florida
   • No; it uses criminal history and questionnaire answers
   • No; it uses only a defendant's age
6. What did the 2018 study by Julia Dressel and Hany Farid find?
   • Untrained online volunteers predicted rearrest about as accurately as COMPAS
   • COMPAS was more than 90 percent accurate
   • Judges ignored COMPAS scores entirely
   • COMPAS had no racial disparities in Broward County
7. Using FPR = (p/(1-p)) x ((1-PPV)/PPV) x (1-FNR), if PPV and FNR are equal across groups but group A has a higher base rate p, what happens to group A's false positive rate?
   • It must be lower
   • It must be zero
   • It is unaffected by p
   • It must be higher
8. Why does leaving race out of a risk model not guarantee unbiased outcomes?
   • Courts require race to be included
   • Correlated inputs such as prior arrests can carry historical policing patterns
   • Models cannot run without demographic data
   • Leaving race out always makes the model less accurate for every group