Evals, or evaluations, are structured tests for measuring the quality of a system, process, or outcome. In AI applications, an eval measures whether a model, agent, or workflow behaved as intended on a specific input, and returns a label, a score, or a written explanation you can act on.
The reason to care is narrow and practical. AI systems fail quietly. A prompt edit, a model version bump, a changed tool schema, or a rebuilt retrieval index can all look harmless in review and still break the path a user depends on. Nothing throws an exception. The output is still fluent, still plausible, and now wrong. Evals are how you find that out deliberately instead of hearing it from a support ticket three weeks later.
Key takeaways
- An eval has three parts: the thing being evaluated, the criteria for judging it, and a scoring method that returns a label or score. Change any one of them and you have a different eval.
- Scoring can be deterministic code, an LLM judge, a human reviewer, or a mix. The choice depends on whether the property you care about can be checked mechanically.
- Offline evals run against a fixed dataset before you ship. Online evals run against live traffic. They answer different questions and most teams need both.
- One eval run is a data point. A practice is repeatable, versioned, wired into CI, and tied back to traces so a bad score points at the step that caused it.
- A green suite is evidence about the criteria you chose, not proof the system is good. Evals only see what you asked them to look at.
The three parts of an eval
The subject. What is being judged: a single model output, an input-output pair with the retrieved context that produced it, a tool call and its arguments, or an entire agent trajectory. The subject sets the boundary of the claim. Scoring a final answer tells you nothing about whether the agent took an expensive or unsafe route to get there.
The criteria. The written definition of what counts as good. For a factuality check this is the rule that an answer must be supported by the retrieved context. For a tone check it is a description of the voice you want. Criteria written as a full rubric, with levels and examples, produce far more consistent results than a one-line instruction.
The scoring method. The mechanism that turns the subject plus the criteria into an output. A hallucination eval typically returns a label such as hallucinated or factual with an explanation attached. Others return a float or a pass/fail. The scoring function might be a schema validator, an embedding distance, an LLM judge, or a human annotation queue.
Offline and online evals
Offline evals run against a fixed evaluation dataset. Inputs are known, the set does not move between runs, and the point is comparison: this prompt against that one, this model against the one you are considering replacing it with. Because the data is stable, a score change means a system change.
Online evals run against production traffic, usually on a sample. There are no reference answers, so scoring is limited to reference-free checks such as groundedness, policy compliance, or structural validity. What you get in exchange is coverage of the inputs real users send, including the ones nobody thought to put in the test set. Offline evals catch regressions before release. Online evals catch the distribution you failed to anticipate.
Why a suite is not yet a practice
Running a few evals is not the same as having an evaluation practice. The difference shows up in the unglamorous parts: the dataset is versioned, scoring functions are reviewed like application code, the suite runs on every pull request, and the results are durable enough to compare against last month.
The other half is diagnosis. A score tells you something failed. It rarely tells you why. That answer lives in the trace of the agent’s steps, where you can see which retrieval returned nothing useful or which tool call came back with an error the model then ignored. Evals without traces produce a number you cannot chase down.
This is also where conventional testing runs out. Unit tests assert on exact values, which covers a narrow band of what an agent can get wrong. Most of the failures traditional tests miss are about judgment, path, and context rather than return values, and they do not reproduce reliably enough for an assertion to hold.
Where evals mislead
The most common problem is measuring what was easy to score instead of what was breaking. Teams reach for correctness because it is familiar, ship a suite that stays at 0.9, and never notice the retrieval step failing on one customer segment. Most eval work that stalls does so before the scoring starts, in the step where someone reads a few hundred real failures and decides what the categories actually are.
The second is treating the aggregate as the result. A mean score of 0.87 across a thousand examples can describe a system that is uniformly decent or one that is excellent on the common case and broken on a slice that matters.
The third is trusting a scorer nobody validated. An LLM judge is a model, and it has its own error rate. Until you have compared its labels against human labels on a sample, an eval score is an opinion with a decimal point on it.
FAQ
What is an eval in AI?
An eval is a structured test that measures whether an AI system behaved as intended. It takes an input and an output, applies criteria, and returns a score or a label, usually with an explanation. The term covers everything from a one-line format check to a human review workflow with a detailed rubric.
What is the difference between an eval and a unit test?
A unit test asserts a deterministic result: this function, this input, this exact output. An eval scores a property of a non-deterministic output, so it returns a measurement rather than a binary pass. You run unit tests to confirm nothing broke. You run evals to see whether quality moved, and by how much, in which direction.
Are evals and benchmarks the same thing?
No. A benchmark is a public, fixed dataset used to compare models against each other on a standard task. An eval is something you build for your own application, against your own data and your own definition of correct. Benchmark scores help you shortlist a model. They tell you very little about whether your system works.
How many evals do I need to start?
Fewer than most teams assume. Two or three that target the failure modes you have actually seen in production will teach you more than fifteen generic ones. Start by reading real failures, group them, and write an eval for the largest group.
Do evals replace human review?
No, and the good ones depend on it. Human labels calibrate an automated scorer: you need them to know whether your judge agrees with you before trusting its output at scale. Most mature setups use humans to audit a sample continuously rather than to review everything.
