Bias AI

Some patterns can produce unfair or harmful outcomes. Evaluating bias requires defining the context and consequences, not merely removing a sensitive column from a dataset.

Key takeaways

• Investigate data and measurement choices.
• Report relevant group results with uncertainty.
• Assess the wider workflow and recourse.

Examine how examples and labels were collected. Missing populations, inconsistent annotation, historical decisions, and selective measurement can shape what the model learns. A target label may reflect an old process rather than the underlying outcome people care about. Measure performance across relevant groups and conditions with suitable privacy controls. Report sample sizes and uncertainty. A small subgroup can have unreliable estimates, while a global average can hide a large and practically important disparity. Different fairness criteria answer different questions and can conflict. Equalizing one statistical measure does not settle every ethical or legal concern. Choose criteria with domain expertise and the participation of people affected by the system. Review the workflow around the model. How predictions are used, who can challenge an outcome, and how feedback is collected can change the distribution of harm. Evaluate mitigations for both their intended effect and possible new problems. Treat fairness as an ongoing assessment rather than a one-time certificate.

1. In an invented test, group A has 900 examples with 95% accuracy, while group B has 100 examples with 60% accuracy.
2. The overall result is dominated by group A. Report group B separately and inspect its errors and sample uncertainty.
3. Investigate data coverage and workflow causes before choosing a mitigation.

These hypothetical counts illustrate why an aggregate score cannot establish equitable performance.