Specificity is the fraction of actual negative cases that the model correctly predicts as negative. It is also called the true negative rate (TNR). For a binary classifier, specificity equals true negatives divided by the sum of true negatives and false positives: TN / (TN + FP). High specificity means the model rarely flags negatives as positives.
If your product cost comes from false alarms on mostly-negative traffic (spam filters with angry users, medical screens with unnecessary follow-ups, fraud alerts that flood analysts), specificity is as important as recall. It answers a different question than precision: not “were my positive predictions right?” but “did I leave the negatives alone?”
Build better agents with Arize
Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.
Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.
Key takeaways
- Specificity = TN / (TN + FP). It measures correct rejection of negative examples.
- Specificity is the complement of the false positive rate: TNR = 1 − FPR when using the same threshold.
- High specificity does not imply high precision; you can avoid false positives on negatives while still generating many false positives if you predict positive often.
- Report specificity alongside recall (sensitivity), precision, and confusion counts at your deployed threshold.
- Choose metrics based on which errors hurt: missed positives (recall) vs false alarms on negatives (specificity or precision).
Where specificity lives in the confusion matrix
Binary classification summaries organize predictions into four counts:
| Predicted negative | Predicted positive | |
|---|---|---|
| Actual negative | True negative (TN) | False positive (FP) |
| Actual positive | False negative (FN) | True positive (TP) |
Recall (sensitivity) = TP / (TP + FN). Of all actual positives, how many did we find?
Specificity (TNR) = TN / (TN + FP). Of all actual negatives, how many did we correctly leave negative?
Precision = TP / (TP + FP). Of all positive predictions, how many were correct?
Specificity pairs naturally with recall when you discuss screening: recall captures missed disease cases; specificity captures healthy patients called sick. Product language maps to the same math for fraud, abuse, and defect detection.
Specificity vs recall: complementary views
Recall and specificity move in opposite directions as you lower the classification threshold. Predict positive more often and recall usually rises while specificity falls, because more negatives get flagged as positives.
The original stub text sometimes described specificity incorrectly. The formula matters: specificity is not “predicted negatives out of all negatives including false positives” in prose that implies precision on negatives. It is strictly the rate at which true negatives are identified among all actual negatives.
When classes are imbalanced, a model can show high accuracy while specificity or recall is unacceptable. Always inspect the full confusion matrix, not one ratio in isolation.
When to optimize for specificity
Prioritize specificity when false positives on negative examples are costly:
- Medical screening follow-ups where false alarms trigger invasive tests.
- Security alerts where analysts have fixed daily capacity.
- Moderation systems that wrongly restrict legitimate users.
- Quality gates where stopping the line on good units wastes throughput.
When missing a positive case is worse (rare disease with treatable early stage, critical fraud), recall usually dominates threshold choice. Many teams set a minimum recall floor, then maximize specificity or precision subject to that constraint.
Teams shipping image classification models with confidence thresholds often plot ROC and precision-recall curves, then pick operating points from business cost rather than default 0.5 cutoffs. Specificity at that point should be reported explicitly when false positives on negatives drive cost.
Specificity, FPR, and ROC
False positive rate (FPR) = FP / (FP + TN) = 1 − specificity. ROC curves plot true positive rate against FPR across thresholds. A model that maintains high specificity at useful recall levels bows toward the upper-left of the ROC plot.
Do not confuse specificity with precision. A model that rarely predicts positive can have high specificity (few negatives flagged) but useless recall. A model that predicts positive for almost everything may have high recall but specificity near zero.
Monitoring specificity in production
Offline eval computes specificity on a labeled holdout set at a chosen threshold. Online, labels arrive late or partially. Practical monitoring includes:
- Rolling confusion counts on audited samples.
- Proxy metrics such as user dispute rate on negative decisions.
- Threshold drift checks when the score distribution shifts but the cutoff stays fixed.
Pair metric tracking with AI model lifecycle management reviews so specificity regressions trigger the same gates as accuracy drops. A drifted feature may leave recall stable while specificity collapses, flooding false alarms.
For LLM classifiers and agent routing, specificity maps to “how often we incorrectly escalate or block benign inputs.” Offline judges and golden sets provide estimates; production needs periodic relabeling because auto-labels go stale.
Specificity in multi-class and LLM settings
Strict specificity is defined for binary decisions. Multi-class extensions usually use one-vs-rest labels per class: for class k, treat “positive” as k and compute TNR among all non-k examples.
LLM safety filters and intent routers often emit scores before a hard decision. Compute specificity on the binary action taken (block vs allow, route vs do not route) at the deployed threshold. For generational tasks without a fixed negative class, specificity is less central than precision-oriented metrics; use it where explicit negative examples exist.
LLM and agent evaluation platforms help keep classical classification metrics and LLM judge scores tied to the same labeled datasets so specificity does not get lost when teams focus only on average rating.
FAQ
What is the difference between specificity and precision?
Specificity conditions on actual negatives: how many did we correctly reject? Precision conditions on predicted positives: how many of our positive calls were right? A model can have high specificity while precision is low if it predicts positive frequently.
Is specificity the same as true negative rate?
Yes. Specificity and true negative rate (TNR) refer to the same ratio: TN / (TN + FP).
Why did my specificity drop after retraining?
Common causes include score distribution shift, a changed threshold, label definition drift, or the new model predicting positive more aggressively. Compare confusion counts at the same threshold on the same eval split before trusting headline accuracy.
Should I report specificity for imbalanced data?
Yes, especially when negatives dominate and false positives are expensive. Also report recall, precision, and raw FP/FN counts so stakeholders see volume, not only rates.
How does specificity relate to the ROC curve?
ROC uses false positive rate on the x-axis, where FPR = 1 − specificity. Each threshold gives a (FPR, TPR) point. High specificity at a operating point means low FPR at that threshold.
