AI in Courts and Judicial Risk Assessment

Risk assessment in criminal justice is older than machine learning. Many tools are fairly simple: a points system or regression model built from historical data. They score factors such as age, prior convictions, prior failures to appear and pending charges. The Public Safety Assessment (PSA), developed with funding from Arnold Ventures, draws on age and criminal history without interviewing the defendant. It produces separate scores for failure to appear and for new criminal activity, plus a flag for new violent criminal activity. COMPAS, sold by Northpointe (now Equivant), is a proprietary tool used in pretrial, sentencing and corrections decisions in some places. It also uses questionnaire answers. In 2016 ProPublica analyzed COMPAS scores in Broward County, Florida. It reported that Black defendants who did not reoffend were nearly twice as likely as white defendants to have been labeled higher risk, and that white defendants who did reoffend were more often labeled lower risk. Northpointe replied that the scores were equally predictive for both groups: a given score matched similar reoffense rates regardless of race. Both claims can be true at once. Researchers including Jon Kleinberg and Alexandra Chouldechova showed that when base rates differ between groups, a score generally cannot be calibrated and also have equal false positive and false negative rates. Choosing a definition of fairness is a policy decision, not a purely technical one. There are other concerns. Rearrest stands in for reoffending, and it reflects where police patrol. Proprietary models make scores harder to challenge. Judges may defer to scores, or override them selectively. Supporters answer that unaided judicial intuition is also biased and less consistent. A common misconception is that these tools are advanced AI. Most are transparent statistical models using a handful of factors. Most other AI in courts is administrative: sorting e-filings, transcription, translation and scheduling.