Often used as a model fairness metric, recall parity measures how sensitive the model is for one group compared to another, or a model’s ability to predict true positives correctly at equal rates across groups. Unequal recall across cohorts means one group experiences more missed positives at the same threshold.
Key takeaways
- Recall parity compares true positive rate (TPR) across groups at a fixed threshold.
- Low recall parity means one group sees more false negatives for the same model settings.
- Different from false positive parity (FPR) and disparate impact (overall pass rates).
- Equalizing recall may require lower thresholds or better features for underserved groups.
- Small groups produce noisy TPR estimates; report counts and confidence intervals.
Definition
For group g, recall (TPR_g) = TP_g / (TP_g + FN_g).
Recall parity holds when TPR_g is equal within policy tolerance for all compared groups.
This is one component of equalized odds, which also requires false positive rate parity.
Why it matters
Missed positives harm differently across groups: denied benefits, undetected fraud targeting a community, or fewer flagged safety issues.
A model with equal approval rates can still fail recall parity if one group has more false negatives hidden inside approved predictions (check confusion cells carefully).
Healthcare, lending, hiring, and moderation contexts scrutinize recall parity when misses disproportionately affect protected classes.
Measurement practice
Fix model version and threshold before comparison.
Use policy-approved group definitions; document legal constraints on attribute use.
Report TP, FN per group, not only rates.
Track over time as base rates and drift evolve.
Mitigation
Collect more labeled data for underperforming cohorts.
Feature improvements targeting signals that work across groups.
Threshold adjustments per group where regulation permits, with governance review.
Post-processing that equalizes TPR at some precision cost.
Lifecycle governance in AI model lifecycle management is where parity metrics belong beside accuracy gates.
Slice deployment practices in shipping image classification models with confidence extend to fairness cohorts.
Log parity metrics with agent traces using LLM and agent evaluation platforms.
Limits
Equal TPR does not guarantee equal harm if base rates or error costs differ.
Optimizing recall parity alone can explode false positives for some groups.
Proxy variables can hide disparities until slice analysis surfaces them.
Equalized odds framing
Recall parity is one leg of equalized odds. Report alongside FPR parity in fairness review packets even when only one metric triggered investigation.
Data collection
Improving recall parity often starts with targeted label collection for underperforming groups rather than immediate architecture changes.
Practitioner checklist
Before changing production settings, confirm labels, thresholds, and monitor windows match the definitions used in your last offline eval. Snapshot dashboards when incidents start so postmortems compare apples to apples.
Run slice-level reviews on high-traffic cohorts weekly even when global metrics look flat. Many failures appear first in one locale, product line, or prompt route.
Document model version, featurizer hash, and data window in every incident ticket. Future you needs that context to interpret drift charts.
When metrics disagree (offline vs online, precision vs recall), write down the business cost of each error type before picking a fix. Metrics are proxies; costs are the decision.
Share eval harness links in release notes so support and PM teams know which golden tasks must pass before they announce improvements.
Related concepts
Recall parity relates to false positive parity and disparate impact reviews. Fairness packets should cross-link the metrics instead of treating each in isolation. Include sample sizes and time windows so reviewers judge statistical noise versus material gaps. Recompute parity after any threshold or model promotion before external fairness reviews.
FAQ
Is recall parity the same as demographic parity?
No. Demographic parity concerns overall positive prediction rates, not true positive detection rates among actual positives.
Can I improve recall parity without retraining?
Sometimes via threshold changes or review queues, but feature fixes often need retrain.
How does recall parity interact with imbalance?
Rare positives per group make TPR volatile; widen windows or pool adjacent weeks carefully.
Do LLM moderation systems use recall parity?
On audited samples with human labels, compare TPR of harmful content detection across cohorts.
How often should I report recall parity?
Each release and on rolling production evals with sufficient positives per group.
