Agents that evaluate agents are evaluators built as agents rather than as a single model call. Instead of receiving a prompt that contains an input, an output, and a rubric, the evaluator can take multiple steps: pull the trace, read the source document the answer cited, call the same API the agent called, run the test suite, and reason across the whole trajectory before it commits to a score.
Build an evaluator this way because a one-shot judge can only reason about what you handed it. If the agent claimed a refund was processed, a single judge call can assess whether that sentence is well formed and consistent with the prompt. It cannot check the refund. An evaluator with tool access can, which moves a class of questions from plausibility judgment to verification.
It matters most where the final answer tells you nothing. A coding agent can produce passing code that ignores every convention in the repository. A support agent can answer correctly after calling an internal API it should never touch. The failure is in the path, and an agent evaluator can look at the path.
Build better agents with Arize
Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.
Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.
Key takeaways
- An agent evaluator is a tool-using, multi-step evaluator. Its distinguishing capability is verification: checking a claim against a source of truth instead of judging whether the claim sounds right.
- It can reason over a full trajectory, which is what makes it useful for agents whose failures are in tool selection, argument construction, and recovery rather than in the final wording.
- The central risk is correlated error. An evaluator built on the same model as the system it grades shares its blind spots, and it will approve exactly the mistakes it would have made.
- An agent evaluator is itself an agent, so it can loop, misuse a tool, or report that it verified something it never checked. Trace it like production software.
- Cost and non-determinism keep this from being a default. Reserve it for cases where verification is possible and valuable, and keep deterministic checks underneath it.
What an agent evaluator can do that a judge call cannot
Verify against a source of truth. Run the generated code and see whether the tests pass. Query the order system and see whether the order exists. Fetch the cited page and check whether it says what the answer claimed. This is the capability that changes the character of the evaluation, because the result stops depending on the judge model’s opinion.
Inspect the trajectory. Read the actual span tree: which tool was called, with what arguments, what came back, what the agent did after an error. Tool arguments in particular are where silent failure lives, and they never appear in the final response.
Take more than one step. Form a hypothesis about why a run failed, check it, revise. That is diagnosis rather than scoring, and it produces something a human can act on. It also removes your obligation to guess in advance what context the evaluator needs, since one that can retrieve will go get it.
The pattern has a name worth knowing, since it is the one most tooling uses: agent-as-a-judge.
Where it goes wrong
Correlated failure. This is the one to take seriously. An evaluator that shares a model family, a prompt style, and a worldview with the system it grades is not an independent check. It is the same reasoning applied twice. Work on whether to use the same LLM for your eval as for your agent points at the practical version of this: self-preference is measurable, and the fix usually starts with using a different model for the judge than for the system.
The evaluator inherits every agent failure mode. It can loop. It can call the wrong tool. It can hallucinate a verification step, producing an authoritative paragraph explaining that it ran the tests when no test span exists in its own trace. An evaluator that fabricates evidence is worse than no evaluator, because its output reads as more trustworthy than a score.
Non-determinism compounds. Two runs of the same agent evaluator on the same input can disagree, and for a reason a judge call does not have: they took different paths. Score variance gets harder to reason about, and small differences between two versions of a system get genuinely hard to detect.
Cost. An agent evaluator can cost more per record than the run it evaluates, since it may make more model calls than the agent did.
Keeping an agent evaluator honest
Treat it as a production component, not as measurement infrastructure that is somehow exempt from measurement.
- Trace the evaluator. Its own runs need spans, the same as the system under test. If you cannot see which tools the evaluator called, you cannot tell verification from assertion. That is a practical argument for an observability platform that covers agent behavior rather than an evaluation script with no visibility of its own.
- Require evidence in the output. Structure the verdict so the evaluator cites what it checked: file paths, tool call IDs, the response it read. Discard unsupported verdicts rather than trusting them.
- Measure agreement against human labels. A few dozen carefully labeled runs is the entire basis on which you get to claim the evaluator works. Rerun that comparison after every change to its prompt or model.
- Use a different model than the system under test where the option exists.
- Keep deterministic checks underneath. Schema validity, forbidden tool calls, and step limits cost nothing and cannot be talked into a favorable interpretation.
When it is worth it
Agent evaluators earn their cost when verification is possible and the answer alone is not sufficient: coding agents where tests can be run, workflows touching a system of record you can query, and high-value low-volume runs where a wrong outcome is expensive.
They are the wrong tool for high-volume scoring of short outputs, for anything scored in the request path, and for criteria a code check already covers. A tiered arrangement usually beats a single choice: deterministic checks on everything, a conventional judge on a sample, an agent evaluator on what the cheaper layers flagged.
FAQ
What is the difference between agent-as-a-judge and LLM-as-a-judge?
LLM-as-a-judge is one model call that reads what you put in the prompt and returns a score or label. Agent-as-a-judge is an evaluator that can act: retrieve documents, call tools, execute code, and step through a trace before deciding. The practical difference is verification. A judge assesses plausibility; an agent evaluator can check the fact.
Can an agent evaluate itself?
It can, and the result should be read with suspicion. Self-evaluation shares the model’s blind spots and tends toward self-preference, so an agent grading its own work will approve the errors it is prone to making. Self-critique is useful as an in-loop mechanism for catching obvious slips, but it is not an independent measurement, and it should not be the number reported to anyone.
How do I know the agent evaluator is right?
The same way you validate any evaluator: label a set of runs by hand, run the evaluator against the same set, and measure agreement. Look closely at the disagreements, since they usually reveal an ambiguous rubric rather than a broken evaluator. Without an agreement number, you have an opinion generator with a tool belt.
