EvalOps is the operational practice of running evaluations continuously across the agent development and deployment lifecycle. It brings CI/CD discipline to AI systems: run evals on changes, detect regressions, notify whoever can fix them, gate risky releases, and route results back into the next iteration.
EvalOps is not “more evals.” It is the workflow layer around them. Evaluation infrastructure is the thing you build, meaning datasets, evaluators, runners, and storage. EvalOps is how you operate it: who owns which evaluator, how often suites run, what blocks a deploy, which score change wakes someone up, and how a production failure becomes a permanent test case.
Build better agents with Arize
Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.
Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.
Key takeaways
- EvalOps is the operating discipline around evaluation, not a tool: ownership, cadence, versioning, gating, alerting, and a feedback loop from production into datasets.
- Evaluator prompts, judge model versions, datasets, and thresholds all need version control. Without it you cannot tell whether a score moved because your system changed or because your evaluator did.
- Gates belong on a small number of metrics you trust. Gating a merge on a noisy judge score produces overrides that become automatic, which is the same as having no gate.
- Route eval alerts to the team that owns the failing service, not to a central eval group, because the fix is almost always in that team’s prompt, retrieval, or tool contract.
- Every production failure that mattered should end up as a dataset row, so the same regression cannot ship twice.
The components of an EvalOps practice
Versioned evaluators and datasets
An LLM judge is a prompt plus a model plus a parsing rule. All three are code and belong in version control, with a version identifier attached to every score you store. Pin the judge model to a specific version rather than a moving alias, because a provider upgrading the model behind that alias shifts your scores with no change to your application.
When you change an evaluator, rescore a fixed baseline set with both versions before shipping the new one. That separates “the evaluator got stricter” from “the system got worse,” which is otherwise hard to answer after the fact.
Evals in CI
The pull request path is where EvalOps most resembles CI/CD. A practical setup runs a fast subset on every pull request, a full suite nightly, and a targeted suite when a prompt or retrieval config changes. The suite writes a comparison against the base branch, so a reviewer sees which cases changed behavior rather than only an aggregate delta.
Gating rules should be boring and few: block on a named metric falling below a recorded threshold on a specific dataset, allow an override that requires a written reason, and never gate on a metric whose run-to-run variance you have not measured.
Continuous online evals and monitors
Once shipped, evaluation runs on sampled production traces, and the discipline you apply to any other production signal applies here: monitor by slice, set thresholds per route, watch the sample size behind each number. Score monitoring sits alongside latency, cost, and error rate as part of what an agent observability platform is for.
Ownership and cadence
The part teams skip. Concretely, name:
- An owner per evaluator. Someone who can say what the score means and reviews it when it drifts.
- An owner per dataset. Someone who accepts or rejects rows harvested from production.
- A review cadence. A weekly pass over recent failures and labels; a monthly check that judge labels still agree with human labels on a held-out sample.
- A release checklist. Which suites must be green, which thresholds apply, who signs off on an override.
Most of this belongs to the day-to-day AI engineering job rather than a separate function, because whoever can fix a bad score usually wrote the prompt or the tool.
Alerting and who gets paged
Split alerts by consequence, not by metric type. A guardrail failure rate spiking on a user-facing route is a page for that service’s on-call engineer. A slow decline in answer groundedness is a digest item for the owning team’s weekly review. A dataset going stale is a ticket.
Two rules keep this survivable. Every alert needs a named owner and a runbook that starts with “open the failing traces,” because a score with no path to an example is not actionable. And distinguish “the eval ran and the system failed” from “the eval could not run,” since those have different responders and conflating them trains people to ignore both.
The feedback loop
Production failures become dataset rows with the expected behavior recorded, usually through a human review step. That is what makes the practice compound: the offline suite grows toward the real distribution instead of aging away from it, and the same regression stops recurring.
How EvalOps differs from CI/CD for ordinary software
Three differences change the mechanics. Tests are not deterministic, so gates compare distributions against thresholds rather than asserting equality. The test oracle is itself a model, so it can be wrong and needs validating against human labels. And every run costs tokens, which makes sampling rate and judge choice engineering decisions rather than configuration details.
An agent’s failure can also sit in retrieval, tool selection, argument construction, control flow, or the assembled context, so the surrounding harness engineering work largely determines how debuggable your eval results are.
Common failure modes
- Alert fatigue. A noisy judge on a low-traffic slice pages someone at 3am for nothing, and within two weeks every eval alert is muted.
- Unversioned evaluators. Scores from March are not comparable to June and nobody can reconstruct why.
- Aggregate-only gating. A mean score holds while one customer segment degrades badly.
- Gates with no debugging path. A red check nobody can trace to failing examples gets overridden by default.
- Cost surprises. Judging every span with an expensive model, discovered at the end of the billing cycle.
FAQ
Is EvalOps just CI/CD for prompts?
CI gating is one component. The rest is production monitoring on sampled traces, alert routing and ownership, versioning, and the loop that turns production failures into regression tests. A team that only runs evals in CI keeps learning about failures from users.
Who owns EvalOps on a team?
Usually the engineers who build the application, with a platform team providing the shared harness. Central ownership of the infrastructure works. Central ownership of what “good” means does not.
What should gate a deploy?
A small set of metrics with known variance, evaluated on a versioned dataset, with thresholds recorded next to that version. Good candidates are regressions on critical-path behavior, safety or policy failures, and a rise in tool call errors. Poor candidates are broad quality averages and metrics that swing between identical runs.
How often should evals run?
Fast subset per pull request, full suite nightly or before release, continuous sampled evals in production, and a human review cadence on top. Adjust by risk: a payments-adjacent workflow warrants stricter checks than an internal summarizer.
Who gets paged when an eval score drops?
The on-call owner of the service that produced the traces, for anything user-facing and sudden. Gradual drifts belong in a review queue instead. Whoever gets the alert needs the failing traces, the slice that moved, and the evaluator version, or they cannot act on it.
