Nhungamiro yehunyanzvi

AI Fairness Metrics: Demographic Parity kune Yakaenzana Odds

Group fairness metrics measure whether a model's decisions or errors differ across demographic groups.

Teerera kugwaro iriAI-yakarondedzerwa · ~3 min

Pa peji ino4 min verenga
  1. Pfupiso
  2. Kudzika Kwakadzika
  3. Kuitwa muNyika yeChokwadi
  4. The Future of AI Fairness Metrics: Demographic Parity to Equalized Odds
  5. Kuzvishandisa
  6. Ramba Uchiongorora
  7. Mibvunzo inowanzo bvunzwa

Pfupiso

Demographic parity compares selection rates, equal opportunity compares true positive rates, equalized odds compares both true and false positive rates, and predictive parity compares precision. These definitions matter because they can conflict mathematically, so choosing one is a value judgment about which kind of unfairness matters most in a particular use case.

Kudzika Kwakadzika

Take a loan model and two groups, A and B, of 100 applicants each. In A, 50 would repay; in B, 20 would. The model approves 40 in A (35 who would repay, 5 who would not) and 20 in B (14 who would repay, 6 who would not). Demographic parity compares selection rates: 40% versus 20%. The ratio of 0.5 falls below the four-fifths (0.8) threshold from US employment guidelines, so it fails. Equal opportunity compares true positive rates (TPR), the share of qualified people approved: 35/50 = 0.70 and 14/20 = 0.70. It is satisfied. Equalized odds (Hardt, Price and Srebro, 2016) also requires equal false positive rates (FPR): 5/50 = 0.10 versus 6/80 = 0.075. Close, but not equal. Predictive parity compares precision, or positive predictive value (PPV): 35/40 = 0.875 versus 14/20 = 0.70. It fails. When base rates differ, as here, an imperfect classifier generally cannot satisfy several definitions at once. Chouldechova (2017) and Kleinberg, Mullainathan and Raghavan (2016) proved versions of this: with unequal base rates, predictive parity or calibration cannot hold alongside equal false positive and false negative rates, except in degenerate cases such as perfect prediction. The COMPAS debate was exactly this conflict. Choosing a metric depends on context. Demographic parity suits cases where the labels themselves may be biased or where the goal is equal access. Equal opportunity suits cases where missing a qualified person is the main harm. Equalized odds adds concern about false accusations. Predictive parity or calibration matters when decision-makers read a score as a probability. A common misconception is that removing the protected attribute makes a model fair. Correlated proxies such as zip code can carry the same information.

Kuitwa muNyika yeChokwadi

A hiring screen advances 40% of one group's applicants and 20% of another's. The ratio of 0.5 falls below the four-fifths rule of thumb used in US employment discrimination analysis, which flags a demographic parity concern.

A medical screening model catches 70% of true cases in both groups. It satisfies equal opportunity, even though the groups' selection rates differ.

ProPublica's 2016 analysis of the COMPAS recidivism tool found Black defendants had higher false positive rates. The vendor responded that its scores had similar predictive value across groups. The dispute showed two fairness metrics in direct conflict.

A data scientist uses Fairlearn's MetricFrame to break accuracy, selection rate and false positive rate down by group. She then applies ThresholdOptimizer to set group-specific thresholds that satisfy equalized odds.

The Future of AI Fairness Metrics: Demographic Parity to Equalized Odds

Group metrics are increasingly written into audits and regulation. New York City's law on automated employment decision tools, for example, requires bias audits that report impact ratios. Research continues on individual fairness, causal fairness, and fairness for generative models, where outputs are text rather than yes-or-no decisions and the standard metrics do not apply directly. The impossibility results are mathematical and will not go away. Future progress is more likely to come from better labels, better documentation of which trade-offs were chosen and why, and stakeholder input than from a single universal metric.

Kuzvishandisa

Nei zvichikosha

  • Mutengo uye bhajeti. Zvisarudzo zvezvivakwa zvinotyaira kuita uye mutengo wekushandisa kwemakore.
  • Sarudzo dzakajeka. Dzidzo yehunyanzvi inobatsira zvikwata kusarudza murwi wakakodzera, kwete iwo mutsva chete.
  • Kudzora kwemhando yepamusoro. Sarudzo dzeinjiniya dziri nani dzinoderedza zviitiko zvekuvimbika mukugadzira.

Chenjera

  • Kugadzirisa imwe bhenji kunogona kuvanza yakafara system kushaya simba.
  • Infrastructure uye mari yekugadzirisa inowanzotarisirwa pasi.
  • Chengetedzo uye kucherechedzwa mapundu anogona kukura sezvo masisitimu anowedzera kuoma.

Matanho anotevera

  1. Tsanangura latency, mhando, uye mutengo zvinangwa usati waitwa.
  2. Benchmark pasi pechokwadi mutoro uye data mamiriro.
  3. Chishandiso chekutarisa zvikanganiso, kudonha, uye mushandisi maitiro.
  4. Gadzirira nzira dzekudzosera kumashure uye dzezviitiko usati wawedzera.

Ramba Uchiongorora

Tsamba yemahara

Tora iyo yemazuva ese AI pfupiso

Nhatu dzakasimbiswa dzeAI nyaya mangwanani ega ega pakati pevhiki, dzakanyorwa neChirungu chakajeka. Yemahara zvachose, hapana ads.

Imwe email zuva rega rega revhiki. Bvisa kunyoresa kamwe chete. Hatimbotengese kana kugovera kero yako.

Zviedze iwe pachako

Take the AI Fairness Metrics: Demographic Parity to Equalized Odds quiz

Mhinduro yepakarepo pamhinduro yega yega, uye chitupa chinogovaniswa chine ID inobatika kana wapedza kosi.

Tanga mibvunzo

Tsigira dzidzo yemahara yeAI. AI Understanding ndeye 501(c)(3) isingabatsiri - hapana ads, hapana paywall, zvachose. Ita chipo

Mibvunzo inowanzo bvunzwa

What is AI Fairness Metrics: Demographic Parity to Equalized Odds?

Group fairness metrics measure whether a model's decisions or errors differ across demographic groups. Demographic parity compares selection rates, equal opportunity compares true positive rates, equalized odds compares both true and false positive rates, and predictive parity compares precision. These definitions matter because they can conflict mathematically, so choosing one is a value judgment about which kind of unfairness matters most in a particular use case.

Mumuenzaniso wechikwereti chegwaro, boka A rine 40% yekusarudza mwero uye boka B rine 20%. Izvi zvinotyora metric ipi?

Demographic parity inofananidza kusarudzwa mitengo. Chiyero che 0.5 chinokundikana uye chinowira pasi pechina-chishanu pachikumbaridzo.

Mukana wakaenzana unoda huwandu hupi kuti huenzane pamapoka ese?

Mukana wakaenzana unobvunza kuti vanhu vanokwanisa vanotenderwa pamwero wakafanana, inova chiyero chechokwadi chakanaka.

Chii chinowedzera mikana yakaenzana pamusoro pemukana wakaenzana?

Equalized odds inoda ese akaenzana echokwadi mareti akanaka uye akaenzana enhema mareti akanaka mumapoka.

Mumuenzaniso, mwero wenhema weboka B i6 pa80. Sei denominator iri makumi masere?

FPR yakaenzana nemanyepo akapatsanurwa neasina kunaka chaiwo. Boka B rine vanyoreri zana uye makumi maviri vaizodzorera, izvo zvinosiya makumi masere chaiwo asina kunaka.

Chii chisingagoneki chinoguma kubva kuChouldechova naKleinberg et al. kureva?

Nezviyero zvisina kuenzana zvepasi, maitiro aya anopokana kunze kwezvinoitika zvakadzikira senge fungidziro yakakwana. Nharo yeCOMPAS yakaratidza kusawirirana uku mukuita.