概述
It became the central case study in algorithmic fairness after a 2016 ProPublica investigation found that its errors fell unevenly by race. The debate that followed showed that common definitions of fairness can mathematically conflict.
深入探討
In May 2016, ProPublica published "Machine Bias." It analyzed COMPAS scores for roughly 7,000 people arrested in Broward County, Florida, in 2013 and 2014, and checked who was charged with a new crime within two years. Race is not an input to COMPAS, which uses criminal history and a questionnaire. Even so, ProPublica found that among people who did not reoffend, Black defendants were nearly twice as likely as white defendants to have been labeled higher risk. Among people who did reoffend, white defendants were more often labeled lower risk. Put simply, false positive rates were higher for Black defendants and false negative rates were higher for white defendants. ProPublica also reported that overall accuracy was modest, around 60 percent. Northpointe replied that the tool was fair by a different standard. At each score level, Black and white defendants reoffended at similar rates, so a given score meant the same thing regardless of race. This property is called calibration, or predictive parity. Each side's numbers were largely correct. Researchers then showed why both could be right. Jon Kleinberg, Sendhil Mullainathan and Manish Raghavan, and separately Alexandra Chouldechova, proved that when two groups have different base rates of the outcome, a score cannot be calibrated and also have equal false positive and false negative rates across groups. The only exception is a perfect predictor. So choosing a fairness definition is a value judgment, not a technical detail. In State v. Loomis (2016), the Wisconsin Supreme Court allowed judges to consider COMPAS at sentencing. It ruled that the score could not be the deciding factor and required written cautions to accompany it. A 2018 Dartmouth study by Julia Dressel and Hany Farid found that untrained online volunteers predicted rearrest about as accurately as COMPAS. A common misconception is that leaving race out removes bias. Correlated inputs such as prior arrests can carry historical policing patterns into the score.
現實世界的實施
In a county that uses COMPAS, a pretrial officer combines a defendant's questionnaire answers with their criminal history. The result is a set of decile scores from 1 to 10 that appears in the report sent to the judge.
After State v. Loomis, a Wisconsin judge who receives a COMPAS report also receives written cautions. They warn that the method is proprietary, that the scores describe groups rather than individuals, and that questions have been raised about racial disparities.
A data scientist auditing a lending model computes both calibration and false positive rates for each group. As with COMPAS, equalizing one breaks the other because the groups' base rates differ.
Researchers following the 2018 Dartmouth study show that a simple model using only age and number of prior convictions predicts rearrest about as accurately as the full COMPAS score.
The Future of COMPAS and Bias in Recidivism Algorithms
Risk assessment tools are still used in many US pretrial and sentencing settings, and the debate continues. Some reform groups back them as more consistent than unaided judgment, while others argue they entrench historical bias. Likely developments include requirements for local validation, public reporting of error rates by group, and access for defendants to how scores are produced, though the pace varies by jurisdiction. The impossibility result is now a standard part of fairness teaching and shapes audits in hiring, lending and healthcare. It does not settle which definition to use. That remains a policy choice about which errors a society is more willing to accept.
付諸實踐
為什麼這很重要
- 風險與安全. 災難性和日常的人工智慧危害都取決於誰了解風險以及誰能夠採取行動。
- 更明確的決策. 民眾和專業素養決定強而有力的安全政策在政治上是否可行。
- 突破炒作. 清晰的解釋可以減少炒作、實驗室公關和模糊道德劇場的影響。
留意
- 將存在風險視為科幻小說,同時能力複合。
- 混淆了表面產品安全與高度自治下的對準。
- 只給非英語和非專業觀眾留下低品質的資源。
後續步驟
- 單獨的產品危害、誤用和失控/失調風險。
- 詢問哪些證據會改變您對時間表和嚴重性的看法。
- 比起行銷主張,更喜歡主要來源和具體評估。
- 確定一條行動路徑:職業、政策、資金或技能——而不僅僅是意識。
不斷探索
常見問題
What is COMPAS and Bias in Recidivism Algorithms?
COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) is a commercial risk assessment tool from Northpointe, now Equivant, that scores how likely a defendant is to reoffend. It became the central case study in algorithmic fairness after a 2016 ProPublica investigation found that its errors fell unevenly by race. The debate that followed showed that common definitions of fairness can mathematically conflict.
在未再犯的被告中,ProPublica 發現黑人被告與白人被告相比有何差異?
ProPublica found a higher false positive rate for Black defendants. Among people who did not reoffend, Black defendants were nearly twice as likely to have been labeled higher risk.
Northpointe 為 COMPAS 辯護的核心是什麼?
Northpointe argued predictive parity, also called calibration: a given score corresponds to a similar reoffense rate regardless of race.
在什麼條件下,不可能性結果指出校準與相等錯誤率無法同時成立?
Kleinberg, Mullainathan and Raghavan, and separately Chouldechova, showed the conflict arises whenever base rates differ and prediction is imperfect.
威斯康辛州最高法院在 State v. Loomis 案中裁定了什麼?
The court allowed COMPAS to be used alongside other factors, with written cautions about its limits, but ruled it could not be the deciding factor.
種族是 COMPAS 的輸入嗎?
Race is not an input. Disparities can still arise through correlated inputs and through differences in base rates.