Precision is the fraction of values that actually belong to a positive class out of all the values which were predicted to belong to that class: precision = true positives / (predicted true positives + predicted false positives). When false alarms are costly, precision tells you how much you can trust positive predictions.
Key takeaways
- Precision = TP / (TP + FP): among predicted positives, how many were correct.
- Low precision means many false alarms; users lose trust in positive flags.
- Precision trades with recall when you move the decision threshold.
- Report precision at the operating threshold used in production, not at 0.5 by default.
- Slice precision on cohorts where false positives hurt most (VIP users, high-value transactions).
Formula and confusion matrix
From a binary confusion matrix at threshold t:
True positives (TP): model predicts positive, label is positive.
False positives (FP): model predicts positive, label is negative.
Precision = TP / (TP + FP). Undefined when the model predicts no positives; document that edge case in dashboards.
Multiclass precision is often computed per class one-vs-rest, then averaged (macro or weighted).
When precision dominates
Spam filters where false positives hide real mail.
Medical screening follow-ups where false alarms stress patients and capacity.
Content moderation false flags on benign speech.
Fraud review queues where analysts time is limited.
Marketing attribution where wasted outreach spend follows false positives.
In these settings, teams accept lower recall to raise precision via higher thresholds or better features.
Threshold tuning
Precision typically rises as threshold increases (fewer positives flagged, often cleaner). Recall falls as threshold rises.
Choose threshold on validation data with explicit costs for FP versus FN.
Plot precision-recall curves instead of relying on a single operating point snapshot.
Monitoring
Track precision with delayed labels on production positives. Sudden drops often trace to drift, threshold changes, or label definition updates.
Pair with positive prediction rate: precision can look stable while volume of flags doubles, flooding reviewers.
Guides on AI model lifecycle management place metric gates before promotion.
Deployment slice practices in shipping image classification models with confidence apply directly to precision-critical classifiers.
Agent layers that emit boolean decisions should log precision proxies in LLM and agent evaluation platforms.
Common mistakes
Optimizing accuracy on imbalanced data while precision on positives collapses.
Reporting training precision on the same rows used to fit thresholds.
Ignoring multiclass confusion where precision on a rare class is the real KPI.
Review queue math
Precision ties directly to analyst load: expected reviews per day equals positive prediction rate times volume. Drop precision without volume drop and queues flood.
Multiclass reporting
Publish precision for each harm-critical class even when macro averages look acceptable.
Practitioner checklist
Before changing production settings, confirm labels, thresholds, and monitor windows match the definitions used in your last offline eval. Snapshot dashboards when incidents start so postmortems compare apples to apples.
Run slice-level reviews on high-traffic cohorts weekly even when global metrics look flat. Many failures appear first in one locale, product line, or prompt route.
Document model version, featurizer hash, and data window in every incident ticket. Future you needs that context to interpret drift charts.
When metrics disagree (offline vs online, precision vs recall), write down the business cost of each error type before picking a fix. Metrics are proxies; costs are the decision.
Share eval harness links in release notes so support and PM teams know which golden tasks must pass before they announce improvements.
Related concepts
Precision pairs naturally with recall and the full confusion matrix. Review all three at the production threshold whenever product changes routing rules or reviewer capacity.
FAQ
How is precision different from accuracy?
Accuracy counts all correct predictions. Precision focuses only on predicted positives.
What is a good precision value?
Depends on review capacity and error costs. Compare to previous models and naive baselines on the same eval set.
How does precision relate to PPV?
Positive predictive value is the clinical name for precision in binary settings.
Can LLM outputs have precision?
When outputs map to discrete positive decisions (flag, approve), yes with labeled audits.
Why did precision drop after deployment?
Check threshold changes, population mix, concept drift, and label pipeline bugs affecting FP counts.
