Pass/fail criteria define the conditions under which an AI output, eval run, experiment, or deployment is considered acceptable. They are the specific thresholds and assertions that turn a score into a verdict.
Good criteria are specific enough to act on. “The answer should be good” is not a criterion. “Fails if the response includes unsupported medical advice, omits required disclaimer text, or contradicts retrieved policy documents” is. The second can be checked by a scorer, disputed by a reviewer, and pointed at a specific line of a specific response.
The hard part is not writing the sentence. It is choosing the number. Most teams pick 0.8 because it looks like a B, ship it, and never touch it again. A threshold chosen that way is not measuring your quality bar. It is measuring how a judge model happens to distribute scores on your particular prompt.
Build better agents with Arize
Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.
Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.
Key takeaways
- Criteria come in four shapes: binary must-pass assertions, absolute score thresholds, deltas against a baseline, and per-slice floors.
- Derive thresholds from measured data. Run the current version repeatedly, look at the score distribution and run-to-run variance, then place the line outside the noise.
- Calibrate against human judgment. The useful threshold is where reviewers start calling the output unacceptable, not a round number.
- Every threshold trades false blocks against false passes. Decide which error you can afford, and give safety criteria zero tolerance.
- A threshold set once and never revisited becomes theater. Record the rationale beside the number and review it when the judge, set, or product changes.
The four shapes a criterion takes
Binary must-pass assertions. Deterministic and unambiguous: the response parses as valid JSON against the schema, the required disclaimer string is present, no forbidden category appears, the tool call includes the required arguments. These are cheap, fast, and have no variance. Write as many criteria in this form as you can.
Absolute score thresholds. Pass rate on a suite must be at or above some value. Simple to state, hardest to choose honestly, and the shape most likely to drift out of relevance as the system improves.
Deltas against a baseline. The candidate must not score more than some tolerance below the current production version on the same cases with the same scorer version. This ages better than an absolute threshold and is the right default for release decisions.
Per-slice floors. The criterion applies per segment: no supported language, customer tier, or intent category falls below its own floor. This exists because an aggregate can hold steady while one segment breaks completely.
Real criteria combine shapes. A workable release bar looks like: all safety assertions pass with zero failures, overall task success does not drop more than 3 percent against baseline, and no individual intent category drops more than 10 percent.
How to actually choose a threshold
- Get a baseline distribution. Run the suite against the current production version several times, not once, so you can see the spread. You now know what normal looks like for this scorer on these cases.
- Measure the noise. The difference between two identical runs is your floor for detectability. If repeated runs of one candidate vary by 4 points, a threshold that trips on a 2 point drop will fail builds for no reason. Judge-based scoring always carries some of this, which is part of deciding when an LLM judge is the right instrument and when a deterministic check fits better.
- Calibrate against humans. Take a sample spanning the score range and have someone label each output acceptable or not. Where the human verdict flips, not a round number, is where your threshold belongs. If the labels do not correlate with the scorer at all, fix the scorer before setting any threshold.
- Pick which error you would rather make. Stricter blocks good releases, looser ships regressions, and no setting avoids both.
- Write down why. The number belongs in versioned configuration next to a note on what it was derived from and when, which is what makes it revisitable instead of a constant nobody dares touch.
Zero-tolerance criteria are different
Some criteria do not get a tolerance band. Prompt injection succeeding, a jailbreak producing prohibited content, leaking another user’s data, or emitting regulated advice without required language are single-failure blocks, and averaging them across a suite hides exactly the case you built the check for. Keep them in their own tier with their own suite, and keep the tests adversarial rather than representative, since normal traffic will not produce them. Sets built around jailbreak attempts and the guardrails that stop them are constructed differently from quality datasets for that reason.
When criteria become theater
The failure is quiet. A threshold set at 0.8 two quarters ago now sits well below where the system normally scores, so it never fires and reports pass on every run, including the ones with real regressions. Or it fires constantly, the team overrides it as routine, and the override becomes part of the release checklist. Both look like a functioning gate in a compliance review.
Three things go stale:
- The judge changed. If the scorer model was updated, 0.8 no longer means what it did when you set it. The threshold is denominated in a currency that got revalued.
- The set changed. Adding hard cases lowers scores with no change in quality, and adding easy ones raises them. Either invalidates comparison to a threshold tuned on the old set.
- The product changed. New capability means new failure categories the criteria say nothing about.
So review the criteria on a cadence, and immediately after any of those three events. Thresholds are release policy and they need an owner, which puts them with the other release controls in managing a model’s lifecycle rather than in someone’s script.
FAQ
How do I author basic pass/fail criteria for a model?
Start from failures you have actually seen, not from a metrics list. For each one, write the narrowest checkable statement that would have caught it, preferring a deterministic assertion whenever the failure is mechanically detectable. Sort the results into must-pass assertions and graded quality metrics, then derive thresholds for the graded ones from a baseline run. Five criteria that map to real failures beat twenty generic ones.
What is the difference between pass/fail criteria and evaluation gating?
The criteria are the thresholds and assertions that decide the verdict. Gating is the mechanism that runs them at a point in your workflow and blocks, allows, or routes the change for review based on the answer. Criteria with no gate are just reporting. A gate without defensible criteria is worse, because it stops releases on a number nobody can explain.
Should a criterion be a score threshold or a binary assertion?
Use a binary assertion when the property is objectively checkable: format validity, required text present, forbidden pattern absent. Use a threshold for genuinely graded qualities such as helpfulness, groundedness, or tone. Converting a graded score into a per-case pass or fail is usually a mistake anyway, since it discards the distribution.
Do I need an explanation for each pass/fail decision?
You need one for every failure a human is expected to act on. A score of 0.3 with no reason attached starts an investigation from scratch. The same score with the judge’s rationale and the span it examined starts from a hypothesis. Have scorers emit a structured label plus a short explanation and store both. It also exposes rubric ambiguity, since the explanations show where the judge is applying a rule you did not intend.
