Create and validate evaluators - Arize AX Docs

Evaluation turns raw traces into measurable judgments of quality. Interpret what your traces reveal, define the criteria that separate a good response from a bad one, and build evaluators, both code-based and LLM-as-a-judge, whose scores you can rely on.

Interpret traces and define success criteria

Read what your traces reveal and establish the criteria that distinguish a good response from a bad one.

Code-based evaluators

Score outputs with deterministic checks for criteria that don’t require a model to judge.

LLM-as-a-judge evaluators

Use an LLM to assess qualities that are difficult to capture in code, and write judge prompts that produce consistent scores.

Evaluation anti-patterns

The common mistakes that make evaluations misleading, and how to avoid them.

Validate your LLM judge

Calibrate an LLM judge against human judgment so you can rely on its scores.

Up next

Experiment, monitor, and export traces

Prove changes with experiments, evaluate production traffic, and turn failures into fixes.

Instrument an agent and review traces Experiment, monitor, and export traces

⌘I