Disparate impact is a quantitative measure of the adverse treatment of protected classes that compares the pass rate, or positive outcome, of one group versus another. In model governance, teams compute selection or approval rates for a protected attribute (gender, race, age band where legally relevant) and compare them to a reference group. Large gaps trigger review even when no single feature explicitly encodes the protected class.
Build better agents with Arize
Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.
Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.
Key takeaways
- Disparate impact compares positive prediction rates between groups, often as a ratio or difference of pass rates.
- It is a statistical screen, not proof of intent or illegality; context and error types matter.
- Proxy features can produce disparate impact without using protected attributes directly.
- Pair rate comparisons with error metrics (false positive and false negative parity) for a fuller picture.
- Document thresholds, cohort definitions, and remediation steps for audit trails.
How it is measured
Define a positive outcome (loan approved, content promoted, fraud cleared). Split predictions by protected group and reference group. Compute pass rate for each.
A common ratio form divides the minority group pass rate by the reference pass rate. Values far from 1.0 warrant investigation. Some frameworks cite informal bands (for example 0.8) as review triggers; legal and policy teams set actual standards for your jurisdiction and use case.
Report confidence intervals when sample sizes are small. Noise can look like disparity.
Relationship to other fairness metrics
Disparate impact focuses on outcome rates.
False positive parity asks whether false alarms are equally likely across groups.
Recall parity asks whether true positives are found at equal rates.
A model can satisfy one metric and fail another. High approval parity with low recall parity means one group sees more missed positives.
Model development implications
Remove protected attributes from feature lists when policy requires, but proxy variables (zip code, school name) may still encode them. Fairness work often needs slice evaluation and sometimes constrained optimization or post-processing.
Collect labels and features needed for slice metrics early. Retrofitting fairness tests after launch is slower and riskier.
Balance fairness goals with utility. Blindly equalizing pass rates without regard to legitimate factors can harm the groups you intend to protect.
Run pre-deployment disparate impact checks on the same validation set and threshold you will use in production. Offline surprises are cheaper than regulatory or reputational incidents after launch.
Monitoring in production
Track pass rates and business outcomes by group over time. Drift in input distributions can change disparate impact without a model redeploy.
Log model version, threshold, and policy rules applied so postmortems attribute shifts correctly.
Guides on AI model lifecycle management include governance checkpoints where fairness metrics belong beside accuracy.
Classification deployment posts such as shipping image classification models with confidence emphasize slice dashboards that extend naturally to demographic cohorts where policy allows.
When classifiers sit in agent pipelines, aggregate fairness metrics with trace evals from LLM and agent evaluation platforms so routing changes do not hide cohort effects.
Limits and caveats
Small cohorts produce unstable rates.
Multiple testing across many slices finds spurious gaps.
Base rates differ across groups; equal pass rates may not imply equal error costs.
Legal definitions vary by region and domain (employment, credit, housing).
Document remediation playbooks when ratios cross review thresholds: threshold adjustments (where legally permitted), feature review, additional human oversight, or model retraining with updated labels.
Store cohort definitions in version control. When product changes who qualifies as a group, historical disparate impact charts need explicit notes so auditors do not compare incompatible cohorts across quarters.
FAQ
Is disparate impact the same as discrimination?
It is a statistical test used in fairness review. Legal discrimination analysis includes process, intent, and business necessity not captured in one ratio.
Should I use 0.8 as a hard rule?
Treat published ratios as starting points for review, not universal law. Align with counsel and product policy.
Can disparate impact improve after removing sensitive features?
Sometimes, but proxies remain. Evaluate outcomes on slices, not only feature lists.
How often should I recompute disparate impact?
At least each model release and on a schedule in production when volume supports stable estimates.
Does disparate impact apply to LLM outputs?
When models gate access, ranking, or moderation, similar rate comparisons on affected cohorts apply. Define positives clearly (shown, approved, answered).
