The precision-recall curve is the correlation between precision and recall at particular cutoff values, with the cutoffs set according to the model. For each threshold on the model’s score (or logits converted to probability), you classify predictions as positive or negative, compute precision and recall, and plot one against the other. The resulting curve shows how much precision you sacrifice when you push recall higher, and vice versa.
If you work on fraud, abuse, rare disease screening, or any classifier where positives are scarce, the PR curve is usually the first plot you open after training. ROC can look excellent while precision at the operating point you actually care about is unusable. The PR curve keeps the rare class in focus.
Key takeaways
- Precision is the fraction of predicted positives that are true positives. Recall is the fraction of actual positives the model found.
- The PR curve traces both metrics as you sweep the decision threshold from strict to permissive.
- When positives are rare, PR curves expose tradeoffs ROC curves hide. AUC-PR drops when the model floods negatives with false positives.
- Choose an operating point from business cost, not from the prettiest curve. The curve shows possibilities; policy picks one threshold.
- Report PR alongside confusion-matrix counts at the chosen threshold so stakeholders see absolute error volumes, not only ratios.
Precision and recall defined
Consider a binary classifier with a scoring function. Pick a threshold t. Predict positive when score >= t, negative otherwise.
Precision = TP / (TP + FP). Of everything you flagged, how many were correct? High precision means few false alarms among your positive predictions.
Recall = TP / (TP + FN). Of all real positives in the dataset, how many did you catch? High recall means you miss few true cases.
Both depend on t. Lower the threshold and recall rises because you predict positive more often, but precision often falls because you include more false positives. Raise the threshold and precision may improve while recall falls because you skip borderline true cases.
The PR curve makes that tension visible. Each point is a (recall, precision) pair for one threshold. Connect them and you see the frontier of achievable tradeoffs for this model on this dataset.
How to build and read the curve
Training produces scores, not hard labels. Sort examples by score descending. Walk down the list, treating each distinct score as a threshold. At each step, compute precision and recall for positives predicted so far.
A model that ranks positives well yields a curve bowed toward the upper-right: high precision even at high recall. A weak model hugging the baseline shows little gain over random ranking.
Area under the PR curve (AUC-PR) summarizes the plot in one number. Unlike ROC AUC, AUC-PR is sensitive to class imbalance. With 0.5% positives, a naive majority-class baseline has high accuracy but low AUC-PR. That is why AUC-PR is a standard reporting metric on skewed tasks.
PR curve vs ROC when positives are rare
ROC plots true positive rate against false positive rate. FPR divides by the count of true negatives, which is huge in imbalanced problems. You can achieve a low FPR while generating thousands of false positives among negatives, and ROC still looks strong.
Precision divides by predicted positives. When positives are rare, flooding the negative pool with false positives destroys precision immediately. The PR curve reflects that pain.
Example: one million rows, five hundred positives. A model fires ten thousand false positives alongside four hundred true positives. Recall is 0.80. Precision is 400 / 10400, under 0.04. ROC may barely move. The PR curve collapses.
Use ROC when classes are balanced or when false positives on negatives are cheap relative to false negatives. Use the PR curve when missing a positive is costly but so is investigating a false alarm, and positives are uncommon. Teams shipping image classification models with confidence often keep both plots during development, with PR driving the final threshold choice on skewed tasks.
Choosing an operating point
The curve is not the decision. Product policy picks a point:
- Investigation budget. If analysts can review five hundred cases per day, choose the threshold that yields at most five hundred predicted positives at acceptable recall.
- Asymmetric costs. False negatives on fraud may cost more than false positives. Set minimum recall first, then maximize precision subject to that floor.
- Multi-stage pipelines. A high-recall, low-precision first stage feeds a second model or rules engine. Report PR for each stage separately.
Mark the chosen threshold on the plot and store it with the model artifact. Recompute the curve on a fixed labeled set whenever the scoring function changes.
PR curves in modern ML and LLM evaluation
Classical tabular models still rely on PR analysis for churn, spam, and defect detection. The same logic applies to LLM classification heads, safety filters, and agent routing decisions that emit a score before acting.
For LLM systems, labels are often noisier and arrive slower. Offline PR curves come from human review or LLM-as-judge labels on a golden set. Online, track precision and recall proxies: user confirmations, appeal rates, downstream task success. Pair metric review with an LLM and agent evaluation platform workflow so offline curves and online counts stay linked.
Multi-class problems extend PR via one-vs-rest curves per class. For the rarest class, a single one-vs-rest PR plot often matters more than a macro-average.
Always compute curves on held-out data. Two models with similar AUC-PR can diverge sharply at the recall region you actually run, so zoom the plot to that range before picking a threshold.
FAQ
What is a good AUC-PR score?
It depends on prevalence and baseline. Compare against a random ranker and against your previous model version on the same split. On highly imbalanced data, AUC-PR below 0.50 may still beat baseline; absolute numbers matter less than relative improvement and precision at target recall.
Why does my ROC look great but PR looks flat?
Class imbalance. ROC is dominated by the huge negative class. Your model may rank most positives above most negatives while still producing too many false positives for precision to be usable. Switch to PR when choosing thresholds for rare events.
How is the PR curve related to the F-score?
The F-beta score combines precision and recall in one number at a single threshold. The PR curve shows the same ingredients across all thresholds. Pick beta > 1 when recall matters more, beta < 1 when precision matters more, then find that point on the curve.
Can I use a PR curve with more than two classes?
Yes. Draw one curve per class using one-vs-rest labels, or micro-average precision and recall across classes. For imbalanced multi-class problems, inspect the rarest class curve directly.
Should I report PR curves in production monitoring?
Offline PR curves belong in training and release reports. Online, log confusion counts and rolling precision and recall at the deployed threshold. Recompute full curves when you refresh labeled evaluation data, not on every hourly batch.
