EvalOps: How to build AI evaluation release gates

Last reviewed September 22, 2026. This was authored by Aryan Kargwal, PhD at PolyMTL.

What is EvalOps?

EvalOps is the practice of running and maintaining AI evaluations across development and production, then using their results to make release decisions. It connects versioned datasets, evaluators, regression tests, release gates, and production feedback for LLM applications and AI agents. The team defines which evidence each change requires, what blocks deployment, and who owns the decision.

An EvalOps record connects the system version under test with the dataset, evaluators, slice-level results, release conditions, decision owner, and final release outcome. The record lets you reconstruct why a version shipped and determine whether later changes still satisfy the same evidence requirements.

Production feedback closes the loop by exposing failures that pre-release tests missed. Those failures can become new regression cases, change protected slices, or force a release policy to become stricter before the next version ships.

How EvalOps turns evidence into a release decision

Start with one workflow and a production baseline. Evaluate a candidate on the same versioned cases, apply the release policy, and save the decision with its supporting evidence. After deployment, use confirmed failures to update the dataset or policy before the next comparison.

This guide uses EvalOps to mean the operational practice around evaluation. An evaluation harness executes cases and returns scores; EvalOps defines when it runs, which results affect deployment, and how production failures change the next test.

Try Arize AX

Build better agents with Arize

Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.

Get started View AX docs

Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.

System changes start a new EvalOps cycle

A new EvalOps cycle starts when a change can affect behavior that the release policy already measures, or when new evidence changes what the team needs to verify before deployment.

Common triggers include:

• Model changes: moving to another model or materially changing inference behavior.
• Prompt changes: changing instructions that affect how the agent interprets or completes a task.
• Tool changes: adding a tool, changing its permissions, or altering how an action is executed.
• Retrieval changes: modifying the knowledge source or retrieval pipeline.
• Workflow changes: changing routing, delegation, handoffs, or execution order.
• Memory or state changes: changing what the agent can retain or use from earlier interactions.
• Evaluation changes: replacing an evaluator, updating the dataset, or changing a release-critical metric.
• Production failures: discovering a failure mode that the existing evaluation set never tested.

Each trigger creates a reason to revisit the existing release evidence. The next evaluation should focus on the behavior that could have changed, while preserving the previous release conditions needed to compare the candidate with the version already in production.

What evidence should an AI evaluation release gate require?

Pre-release evaluation should show whether the candidate still satisfies the behaviors the release policy cares about. For agents, that evidence often extends beyond the final response into the actions and state changes that produced it. The agent evaluation workflow can combine task-level and trajectory-level checks when the release condition depends on what happened during execution.

Aggregate scores can hide the regression you care about

An average success rate can hide both a critical failure category and inconsistent results across repeated attempts.

The Arize tutorial on building agent evals from traces uses a 75% per-attempt success rate to illustrate how different repeated-run metrics can produce very different interpretations:

The same agent can look highly capable under pass@10 and inconsistent under pass^10. Use pass@k when the workflow can safely retry and reliably select a successful result. For an unattended agent that changes external state, also measure repeated-run consistency and critical failures; a later success does not undo an earlier incorrect action.

These figures assume independent attempts with the same per-attempt success probability. Real agent failures can cluster, so the formulas illustrate different reliability questions rather than forecast production success. One run per example estimates single-attempt performance; repeated runs are needed to estimate consistency on the same cases.

Slice-level thresholds solve a different averaging problem. A candidate can improve overall while regressing on a small high-risk workflow. If that slice is release-critical, give it its own no-regression or minimum-performance rule instead of letting the global score compensate for it.

The evaluator can pass the wrong outcome

Even the right metric can be wrong if its evaluator cannot observe the evidence needed to decide it. The evaluator choice differs across cases:

• Task completion that ends in a state change: verify the resulting system state with code or another deterministic check.
• Tool use: inspect tool selection and arguments at the action level instead of judging only the final answer.
• Trajectory requirements: use a trajectory evaluation when order, retries, approvals, or recovery behavior matters across the trace.
• Semantic criteria: use a calibrated LLM judge for requirements such as relevance or helpfulness. Give the judge the evidence needed to apply its rubric, and check its labels against human-reviewed examples.

For an AI agent that changes customer permissions, a final-response judge can pass a run even when the underlying account state remains unchanged. The evaluator’s scope belongs in the decision record because a PASS without that context does not tell a reviewer what was actually proven.

How to set AI evaluation release gates

A release policy maps evaluation evidence to a deployment action. HOLD blocks a candidate that violates a release-critical requirement. REVIEW pauses the decision until an owner resolves a tradeoff or missing evidence. SHIP means the candidate has cleared every required evaluation gate; other deployment checks still apply.

Set blocking and review conditions

Blocking conditions encode failures that should end the release regardless of gains elsewhere. For the permission-changing agent, a false success after a state-changing action can be an automatic hold, while a regression on the protected permission slice can block the candidate even if its overall score improves. Review conditions are useful when the evidence exposes a tradeoff the policy cannot resolve automatically, such as a slower candidate with meaningfully higher task completion.

Example: A better overall score can still trigger HOLD

Illustrative results, not a production benchmark: The baseline completes 442 of 500 tasks (88.4%), while the candidate completes 463 (92.6%) on the same cases. On the protected permissions slice, the baseline completes 38 of 40 cases and the candidate completes 34 of 40.

Two candidate runs expose a more serious failure. In each, the agent reports that a permissions change succeeded, but an independent account-state check shows that it did not. The evaluator records the case ID, initial permissions, requested change, tool call and response, observed post-action state, and a false_success result.

The release policy allows zero confirmed false successes, so the decision is HOLD. The improvement across all 500 cases cannot override that condition. The permissions-slice regression also warrants investigation, although these counts alone do not establish its statistical significance. Save the failing case IDs and evidence with the release decision.

A simple release policy in code

The evaluation job should produce a versioned result artifact before the release gate runs. For the permissions example, the gate needs the baseline and candidate results, the number of comparable permissions cases, any confirmed false successes, and the uncertainty estimate used for the slice comparison.

An illustrative input looks like this:

{

  "evidence": {

    "complete_and_comparable": true,

    "false_successes": 2,

    "permissions_n": 40,

    "permissions_delta_ci": [-0.24, 0.04],

    "p95_latency_ms": 1800

  },

  "policy": {

    "max_false_successes": 0,

    "min_permissions_n": 40,

    "max_permissions_drop": 0.02,

    "max_p95_latency_ms": 2000

  }

}

The confidence interval is illustrative. The evaluation job must calculate it from the paired baseline and candidate results; it cannot be inferred from the summary counts above. Repeated attempts on one case should not be counted as independent cases.

The release decision function then applies the policy:

def release_decision(evidence, policy):

    if not evidence["complete_and_comparable"]:

        return "REVIEW"
    if evidence["false_successes"] > policy["max_false_successes"]:

        return "HOLD"
    if evidence["permissions_n"] < policy["min_permissions_n"]:
        return "REVIEW"

    low, high = evidence["permissions_delta_ci"]
    boundary = -policy["max_permissions_drop"]

    if high < boundary:
        return "HOLD"

    if low < boundary:
        return "REVIEW"

    if evidence["p95_latency_ms"] > policy["max_p95_latency_ms"]:

        return "REVIEW"
    return "SHIP"

An upstream validation step must confirm that required results exist, numeric values are finite, thresholds are valid, the confidence interval is ordered, and the dataset and evaluator versions are comparable. If that validation fails, set complete_and_comparable to false. Missing or invalid evidence produces REVIEW, which pauses deployment; it does not silently pass the gate.

For this example, the two confirmed false successes produce HOLD before the other conditions are evaluated. A permissions regression of two percentage points is expressed as max_permissions_drop: 0.02. Choose the minimum case count and comparison method based on the regression the team needs to detect, then version those choices with the policy.

In CI, treat both HOLD and unresolved REVIEW as blocking results:

decision = release_decision(evidence, policy)

save_release_record(decision, evidence, policy)
if decision != "SHIP":

    raise SystemExit(f"Release gate blocked deployment: {decision}")

A named owner can investigate a REVIEW result and record an exception under the team’s release process. The CI job should not turn REVIEW into SHIP merely because a result is missing or a threshold is uncertain.

What to record for every EvalOps release decision

Once the policy runs, the release record should preserve enough context to reconstruct the decision later. A reviewer should be able to tell exactly which candidate was evaluated and which evidence authorized its deployment.

• Candidate version and a short description of the change.
• Evaluation dataset version, including any protected slices used for the decision.
• Evaluator version and scope for each release-critical result.
• Results and sample counts that triggered the policy, including the baseline, uncertainty estimate, and links to failing cases when comparison matters.
• Release policy version and the condition that resolved the decision.
• Production evidence used in the decision, such as a trace or confirmed incident.
• Decision owner and timestamp.
• Exception rationale when an owner accepts a review condition or overrides the default action.

This record becomes the reference point for the next EvalOps cycle. When the system changes again, teams can see which evidence needs to be regenerated and which release conditions still apply.

A minimal release record

This illustrative JSON record can accompany the result artifact. Replace the placeholders with the responsible reviewer, timestamp, and immutable evidence location.

{

  "candidate": "permissions-agent-v24",

  "baseline": "permissions-agent-v23",

  "dataset": "access-regression-v7",

  "evaluator": "account-state-check-v3",

  "policy": "release-policy-v2",

  "decision": "HOLD",

  "trigger": "2 confirmed false successes in 40 permissions cases",

  "decision_owner": "",

  "decided_at": "",

  "evidence": ""

}

Production feedback changes the next EvalOps cycle

Production feedback tests whether the assumptions behind the release still hold once the agent sees live traffic. A failure becomes useful to EvalOps when the team can reconstruct it well enough to change the next evaluation decision.

Sampled online evals continue the measurement after deployment. Run release-critical evaluators on an appropriate sample of live traces and monitor protected slices alongside sample counts. Route confirmed critical failures to the service owner for investigation and containment or rollback under the runbook. Distinguish a failing system from an evaluator that could not run.

Observability supplies production evidence

Agent observability preserves the execution context around a production failure. For the permissions agent, instrument the request, tool action, and authoritative post-action state so the trace shows whether the change took effect. If the trace lacks that state, fetch it through an explicit check before concluding that the task succeeded.

Turn production failures into regression cases

A confirmed production failure should become a reproducible case rather than a one-off incident. Preserve enough trace and state evidence to reproduce the failure, then label the expected behavior. After sensitive data is removed, add the case to the regression dataset. Arize AX can create datasets from spans so future experiments replay the same failure against each candidate. If the failure is release-critical, promote it into a protected slice or blocking condition before the next release.

For state-changing workflows, also preserve a resettable test fixture or initial-state snapshot. Each candidate must start from the same conditions; replaying a logged request against an already-modified account can hide the original failure.

Evaluator changes can move the release baseline

An evaluator is part of the measurement system behind a release decision. A new evaluator version can move the result even when the agent itself has not changed. Treat evaluator changes as changes to release evidence when they affect any of the following:

• Judge model or provider: the same rubric can produce a different label or score distribution. Pin a model version where the provider supports it, and record the model identifier and configuration used for each run.
• Rubric or evaluator prompt: a wording change can redefine what the evaluator accepts.
• Evaluator scope: moving from span to trace or session changes the evidence available to the evaluator.
• Variable mappings: a correct rubric can still score the wrong fields when its inputs are remapped.
• Labels or score semantics: changing the output scale changes what an existing release threshold means.
• Code evaluator logic: a deterministic check can move from pass to fail after implementation changes.

Calibrate a new evaluator before it controls release decisions

Before evaluator v2 replaces v1, run both against the same human-labeled calibration set. Arize’s guidance on aligning evals to human feedback recommends roughly 50 to a few hundred examples, including edge cases, as a starting point. Compare accuracy and per-label precision and recall, then inspect release-critical slices separately. After tuning the rubric, validate the revised evaluator on held-out examples.

Revalidate any release threshold tied to the previous evaluator. A score of 0.85 under v1 only remains a valid gate under v2 if the new evaluator preserves the meaning of that boundary. Keep the previous evaluator attached to historical runs, compare v1 and v2 during the transition, and promote the new version after its threshold and failure behavior are understood.

Apply the same calibration process to a newly created evaluator. Before it can block a release, confirm which evidence it can inspect, which requirement it measures, and how its false positives and false negatives affect the decision.

How Arize supports EvalOps

Arize AX connects evaluation work before release with evidence from production. Datasets and experiments let you replay the same cases against a baseline and candidate, then compare the results before deciding what moves forward.

For agents, that comparison can include the trajectory behind the outcome. Traces preserve tool calls, retrieval, latency, cost, and other execution details that help explain why one candidate behaved differently from another.

Production failures can feed the next evaluation cycle. Signal surfaces recurring issues from production traces, while confirmed failures can be saved into datasets and replayed against later candidates.

That workflow can also start from the developer environment. Arize Skills let coding agents work with traces, datasets, experiments, and evaluators, so a failure discovered during development or production can move into the evaluation workflow without rebuilding the process manually.

For nuanced criteria that need context across a trace, Agent-as-a-Judge can evaluate exported trace data using a Claude Code harness. It is currently a closed Enterprise beta. Use deterministic checks for verifiable requirements such as post-action account state, and calibrate any judge before using its results in a release gate.

The release decision still comes from the policy you define. AX provides the traces, datasets, evaluator versions, experiments, and production evidence needed to determine whether the candidate has actually satisfied it.

Related reading

• What is EvalOps?
• How to evaluate AI agents
• LLM-as-a-judge
• What is an evaluation harness?
• Build agent evals from traces
• CI/CD for LLM apps
• Agent observability
• AI agent regression testing
• Offline vs. online evaluation
• LLM evaluation

Frequently asked questions about EvalOps

How is EvalOps different from LLMOps and MLOps?

EvalOps focuses on maintaining AI evaluations and using their results in development and production decisions. MLOps covers the broader operational lifecycle of machine-learning systems, while LLMOps includes concerns such as prompts, retrieval, model configuration, and monitoring for LLM applications. EvalOps supplies the evaluation evidence, release rules, and feedback process within that broader lifecycle.

Does every AI change require the full evaluation suite?

Not necessarily. Rerun the evaluations that can detect behavior affected by the change, plus any standing release-critical regression checks.

A retrieval change may need retrieval and end-to-end outcome evaluation. A tool-schema change may need tool-selection, argument, and resulting-state checks. EvalOps makes the required evidence explicit before the candidate reaches the release gate.

How large should an evaluation dataset be before release?

Dataset size depends on coverage, the smallest regression you need to detect, and the uncertainty you can accept. Include normal tasks, protected slices, and known failure modes, then check sample counts within each critical slice. A large dataset can still leave too few permissions cases to support a release decision.

Treat the dataset as versioned release evidence. Confirmed production failures should become regression cases, and new system changes should add coverage where they introduce behavior the current set cannot test. The useful question is whether the dataset covers the release decision the team needs to make.

Can an LLM-as-a-Judge be used as a blocking release gate?

Yes, when the judge is calibrated for the criterion it controls and its failure modes are understood.

Validate it against trusted human labels and inspect release-critical slices separately. Preserve the evaluator version, scope, and threshold used for the decision.

Use deterministic checks when the requirement can be verified directly, such as post-action state, tool arguments, or approval ordering. Use the LLM judge for semantic criteria it can observe reliably.

How do you know an AI agent is ready for production?

Clearing the evaluation release policy is one requirement for production readiness. Teams also need the permissions, security checks, monitoring, rollback procedures, and operational ownership appropriate to the workflow.

For a state-changing or autonomous workflow, the release evidence should make these checks inspectable:

• verified task outcome in the system where the action occurred
• release-critical failures and protected slices
• repeated-run reliability when stochastic behavior affects risk
• latency or cost against the operating budget
• candidate, dataset, evaluator, and policy versions, plus the decision owner

That record gives the team a baseline for investigating production failures and deciding what the next candidate has to prove.

How is EvalOps different from an evaluation harness?

The evaluation harness executes cases, records traces, and returns scores. EvalOps defines the operating process around it: versioning datasets and evaluators, applying release policy, recording decisions, and turning production failures into regression cases. A harness supplies the execution mechanism; the team still has to define and maintain the release requirements.

Where does EvalOps run in CI/CD?

Run a fast, relevant subset on every pull request and the required release suite before deployment. After deployment, run sampled online evaluations and investigate confirmed failures for inclusion in the regression dataset. The release job should read a versioned result artifact, apply the policy, save its decision, and block deployment on HOLD or unresolved REVIEW.