Intersectional Bias in AI

Intersectionality describes how social categories and systems of disadvantage can interact. In machine learning audits, an intersectional disparity is a performance or outcome difference for a combined subgroup that may disappear when results are aggregated by one attribute at a time. It is not enough to test “race” and “gender” separately if the largest error affects a particular race-by-gender group. The concept also cautions against treating categories as independent or interchangeable variables. Buolamwini and Gebru’s Gender Shades study examined three commercial gender-classification systems using a benchmark balanced by gender and skin type. The tested systems had their highest misclassification rates for darker-skinned women, up to 34.7%, while the maximum for lighter-skinned men was 0.8%. The study also found that two existing benchmarks were overwhelmingly lighter-skinned. These results show why subgroup composition and intersectional evaluation matter, but they apply to the tested systems, data, and binary gender-classification task at that time. They do not establish the accuracy of current products or validate gender inference as appropriate. Single-axis audits can mask compounded effects in hiring, lending, health care, and content moderation. A model can satisfy a threshold for each broad category while failing a smaller intersectional group. Evaluation requires enough examples to estimate performance and uncertainty; small samples can make metrics unstable and may expose sensitive attributes. Teams should justify which intersections are relevant, protect privacy, consult affected communities, and avoid presenting sparse estimates as definitive. Mitigation can include collecting better data with consent, improving measurement, changing the target or workflow, and testing multiple performance measures. No single metric captures every form of harm. Document limitations and provide appeal or human review for high-impact decisions. Intersectional analysis is a continuing practice, not a one-time check before release.