GUIDA della Società

Pregiudizio razziale negli algoritmi sanitari

A 2019 Science study found racial bias in a widely used population-health algorithm because it predicted future healthcare costs as a proxy for health need.

Ascolta questa guidaNarrato dall'IA · ~3 min

In questa pagina3 minuti di lettura
  1. Panoramica
  2. Immersione profonda
  3. Implementazione nel mondo reale
  4. The Future of Racial Bias in Healthcare Algorithms
  5. Metterlo in pratica
  6. Continua a esplorare
  7. Domande frequenti

Panoramica

At the same risk score, Black patients were sicker than White patients; changing the proxy increased Black patients selected for extra support from 17.7% to 46.5% in that study. The result concerns a specific algorithm and sample, not every healthcare model.

Immersione profonda

Obermeyer and coauthors’ 2019 Science study examined a commercial algorithm used to identify patients with complex health needs for extra care-management support. The tool predicted future healthcare spending as a proxy for health need. The authors found that, at a given risk score, Black patients were considerably sicker than White patients, as measured by uncontrolled illness. The study’s mechanism was structural: because Black patients had historically received less care and generated lower costs at similar levels of illness, a cost-prediction target understated their need. The proxy appeared predictive of spending while encoding unequal access to care. In the study population, replacing the cost target with a measure more closely related to illness would have increased the proportion of Black patients receiving additional help from 17.7% to 46.5%. That figure describes the study’s reallocation result, not a nationwide estimate or a general expected correction rate. The research affected an algorithm used across multiple health systems and drew attention to how proxies can transmit inequity without explicitly including race. It did not show that every use of cost data is biased or that the same pattern appears in every model. The broader lesson is to examine whether a target variable represents the desired construct. Expenditure reflects illness, but also insurance, access, clinician decisions, service availability and historical patterns. A model can be statistically accurate for its proxy and still fail the operational objective of identifying people who need care. Follow-up audits should assess patient-level health outcomes, access to interventions and subgroup effects, while recognizing that the 2019 result is specific to one algorithm, intended use and dataset.

Implementazione nel mondo reale

A health system audits whether historical spending reflects unequal access to care before using cost to allocate extra disease-management support.

A model team compares predicted cost with clinical need and outcomes across racial groups rather than treating spending as a neutral health label.

A reviewer checks whether a risk score under-selects Black patients who have comparable or greater illness burden.

A governance committee repeats an evaluation after changing the target from cost to a clinically grounded measure of need.

The Future of Racial Bias in Healthcare Algorithms

The study remains an important example of proxy-target bias, while later systems and datasets require their own evaluation. Health systems should periodically recheck targets, access patterns and outcomes as models or care programs change; do not generalize a 2019 result to a different algorithm without evidence. Keep a dated record of the primary source or study behind each claim and revisit conclusions when new evidence or implementation details emerge. As care pathways and insurance coverage change, cost and utilization data may shift independently from underlying disease burden.

Metterlo in pratica

Perché è importante

  • Rischio e sicurezza. I danni catastrofici e quotidiani dell’IA dipendono entrambi da chi comprende i rischi e da chi può agire.
  • Decisioni più chiare. L’alfabetizzazione pubblica e professionale determina la possibilità politica di una forte politica di sicurezza.
  • Tagliare il clamore. Spiegazioni chiare riducono la cattura da parte di montature pubblicitarie, PR di laboratorio e vaghi teatrini etici.

Attenzione

  • Trattare il rischio esistenziale come fantascienza mentre le capacità si aggravano.
  • Confondere la sicurezza del prodotto superficiale con l'allineamento in condizioni di elevata autonomia.
  • Lasciando il pubblico non inglese e non esperto solo con fonti di bassa qualità.

Prossimi passi

  1. Separare i rischi di danni al prodotto, uso improprio e perdita di controllo/disallineamento.
  2. Chiedi quali prove cambierebbero la tua opinione sulle tempistiche e sulla gravità.
  3. Preferire fonti primarie e valutazioni concrete alle affermazioni di marketing.
  4. Identifica un percorso d’azione: carriera, politica, finanziamenti o competenze, non solo consapevolezza.

Continua a esplorare

Newsletter gratuita

Ricevi il briefing quotidiano sull'IA

Tre storie verificate sull'IA ogni mattina nei giorni feriali, scritte in un inglese semplice. Gratuito per sempre, senza pubblicità.

Un'email ogni giorno feriale. Disiscriviti con un clic. Non vendiamo né condividiamo mai il tuo indirizzo.

Mettiti alla prova

Take the Racial Bias in Healthcare Algorithms quiz

Feedback immediato su ogni risposta e un certificato condivisibile con un ID verificabile una volta superato un corso.

Inizia il quiz

Sostieni l'istruzione gratuita sull'IA. AI Understanding è un'organizzazione non profit 501(c)(3) — nessun annuncio, nessun paywall, mai. Fai una donazione

Domande frequenti

What is Racial Bias in Healthcare Algorithms?

A 2019 Science study found racial bias in a widely used population-health algorithm because it predicted future healthcare costs as a proxy for health need. At the same risk score, Black patients were sicker than White patients; changing the proxy increased Black patients selected for extra support from 17.7% to 46.5% in that study. The result concerns a specific algorithm and sample, not every healthcare model.

What did the studied population-health algorithm predict as a proxy for health need?

The study reports the algorithm predicted healthcare costs, which were used as a proxy for health need.

At the same risk score, what did the researchers observe about Black patients?

The paper found Black patients were sicker at a given score, based on signs of uncontrolled illness.

Why did predicting costs understate Black patients’ health needs in the studied setting?

The authors link the proxy’s bias to lower spending on Black patients relative to White patients with similar illness.

What happened to the share of Black patients identified for extra help when the study’s target was changed?

The authors report the proportion would rise from 17.7% to 46.5% under the alternative target.

What was the key proxy mismatch in the study?

The paper identifies cost as a proxy for health, which did not capture unequal access and illness burden.