Agent reliability: how to measure and improve AI agents in production

An agent can return 200 OK, produce a fluent confirmation, and close a clean session while still failing the user. The refund may be wrong, the requested file may never exist, or the agent may have dropped a constraint halfway through the run.

Agent reliability is whether the complete model-plus-harness system repeatedly completes the intended task under real conditions, preserves the constraints that matter, and leaves valid evidence that the outcome exists.

The outcome has to survive the workflow. The agent has to complete the task, preserve the user’s intent, leave external systems in a valid state, stay within its operating budget, and recover from ordinary interruptions. One clean trajectory proves little in a nondeterministic system, so reliability has to be measured across repeated runs and production traffic.

What is agent reliability?

Agent reliability is a property of the complete model-plus-harness system. A reliable agent repeatedly completes the task, preserves its constraints, and leaves evidence that the outcome exists. It avoids claiming success when the work is incomplete, wrong, unsafe, or impossible to verify.

Model quality describes what a model can do under defined test conditions. Product quality asks whether the shipped experience works. Agent reliability asks whether the completed workflow remains correct when the model interacts with tools, data, users, state, and a changing environment.

The result lives in the changed record, created artifact, resolved conversation, safe handoff, or verified external action. The final answer is evidence only when the answer itself is the task.

Recent research on the science of AI agent reliability argues that a single task-success score hides important operational properties. Reliability also depends on whether an agent behaves consistently across runs, remains robust under perturbation, fails predictably, and keeps the severity of failures bounded.

Most engineering that produces this property belongs to harness engineering. Agent evaluation judges a run. An agent observability platform exposes the evidence. The anatomy of an agent harness describes the runtime around the model. Uber’s production agent evaluation work shows why live failures escape pre-release checks. The AI engineering discipline turns those findings into changes.

For a concise definition, see agent reliability in our glossary.

Why service metrics and model benchmarks miss agent failure

Service health tells you whether the system responded, but not whether the user’s job was successful.

Every span can finish cleanly while the agent updates one of three required records, ignores a retrieved policy, loops until a cap, or announces a result that never reached the underlying system. A model benchmark can show that the model knows how to perform each step without proving that the assembled workflow completes the task repeatedly.

The most damaging failures are often quiet:

• • False completion: The agent reports success without valid outcome evidence.
  • No-progress behavior: The agent repeats tools or reasoning without moving the task forward.
  • Partial completion: The agent performs only part of the requested change and leaves inconsistent state.
  • Constraint loss: The agent finishes a different job after dropping a requirement, permission boundary, or delivery instruction.
  • Harmful success: The agent achieves the objective through an action or path the user did not authorize.

Longer trajectories multiply the problem. If an eight-step task requires every step to succeed and each step has an independent 95% success rate, the chance all eight succeed is about 66%. The independence assumption is only illustrative, but the engineering point holds: high component scores can coexist with weak task-level reliability.

Nondeterminism makes a single pass weak evidence. The same request can produce a different plan, tool path, or recovery decision on the next run. Long-horizon reliability therefore requires repeated trials, varied starting states, and production evidence.

Uber saw this gap in a voice agent that booked rides. A mother asked the agent to book a ride to San Francisco International Airport while her child said they wanted pizza. The agent treated the background comment as a new instruction and began rerouting toward a restaurant. Offline evals missed it. Production exposed the failure when sessions that usually took four or five turns stretched to 15 or 20.

What to measure for agent reliability

Agent reliability is not one number. When we speak with customers and organizations, we typically recommend a holistic view that measures the following:

Task success rate

Did the agent complete the task, preserve constraints, and leave valid evidence? Grade both the trajectory and the outcome.

Repeated-run consistency

How often does the same task continue to succeed across repeated trials, seeds, or semantically equivalent inputs?

Recovery rate

After an error, interruption, stale state, or changed environment, how often did the agent recover and finish safely?

Robustness under perturbation

Does the workflow still work when inputs, tool responses, timing, interface state, or context differ from the happy path?

False-completion and no-progress rate

How often did the agent claim success without evidence or continue acting without advancing the task?

Severity and blast radius

How consequential are failures involving money, permissions, customer records, security boundaries, or public artifacts?

Cost and latency per completed task

What did successful work cost, and how long did it take after retries, recovery, and failed attempts are included?

Escalation, abandonment, and human repair

How often did the agent hand off, get abandoned, or require someone to repair the state it left behind?

Set a reliability budget for each meaningful workflow or traffic slice. Define a task-success floor, a maximum false-completion rate, hard limits for high-severity failures, and tolerances for cost, latency, escalation, or recovery before a regression appears.

Do not collapse frequent low-impact misses and rare high-impact failures into one average. A research assistant and an agent issuing refunds should not share the same threshold. Slice by customer, workflow, language, tool path, run length, model or harness version, and consequence.

Use trace-level evaluation to judge the trajectory, but keep the production scorecard small enough to guide release and operating decisions.

Reliability looks different by agent type

The general property stays the same, but the proof changes with the job. Each agent type has a distinctive way to look successful while leaving the intended outcome false.

Transactional agents fail when the recorded state does not match the request

For a transactional agent, the state left behind is the proof of reliability; the confirmation message is only a claim. A booking, refund, account update, or order change succeeds when the correct record exists in the correct state and related records remain consistent.

Compare the requested change with the resulting state. Check identifiers, amounts, permissions, and read-after-write results. For retried actions, verify that the agent did not create a duplicate charge, reservation, or update. A polished receipt cannot repair a transaction that never committed or committed twice.

Long-horizon agents fail when progress drifts from the original intent

For a long-horizon agent, reliability means preserving the task contract while the plan, context, and environment change. Progress is not enough when the agent quietly abandons an original constraint.

OpenAI’s 2026 work on long-horizon models described an internal agent that was instructed to report through Slack. The agent kept working, found a path around its environment, and published the result through a public GitHub pull request instead. In other words, the run advanced while violating a core instruction.

The proof covers the trajectory and final artifact. Critical constraints should survive context compression, retries, handoffs, and new tool output. The work should match the original delivery channel, scope, permissions, and acceptance criteria. A long run that finishes the wrong job is still a failure.

Computer-use agents fail when they cannot recover from the environment they find

For a computer-use agent, reliability means reorienting from the current interface state. Pages load differently, sessions expire, dialogs appear, and prior work remains half-complete.

Test workflows from varied starting states and interrupt them mid-flow. The agent should inspect the environment, recognize completed work, resume safely, and avoid repeated side effects. Completion is the final application state, not a click sequence that worked once.

Customer-facing agents fail when offline scores stop predicting user outcomes

For a customer-facing agent, reliability means that offline evaluation continues to predict what users experience in production. A rising evaluator score with flat resolution, satisfaction, or self-service usually means the team is improving a proxy.

Nubank’s 2026 work on customer-support agents connected offline simulation with large-scale A/B testing across five production deployments. In its card-delivery use case, a newer agent improved transactional NPS by 37 percentage points and self-service rate by 29 percentage points, while offline metrics moved in the same direction. The important result is the relationship: the offline signal earned trust because it predicted the online outcome.

Recheck that relationship as customer intent, policy, language, and traffic change. Evidence may include resolved cases, self-service completion, satisfaction, repeat contact, correction, abandonment, and escalation. Choose the measure that matches the job the customer came to complete.

Coding agents fail when a passing patch leaves a worse codebase

For a coding agent, reliability means solving the issue while leaving a change that the team can safely review, maintain, and extend. Passing the current test suite is necessary evidence. It does not prove that the patch belongs in the codebase.

A coding agent can satisfy a test by duplicating logic, changing unrelated files, weakening a security boundary, or ignoring repository conventions. Inspect the scoped diff, required tests, dependency changes, static checks, security constraints, and fit with the surrounding architecture. The state left behind is the repository, not the agent’s explanation.

Voice agents fail when timing breaks an otherwise correct answer

For a voice agent, reliability includes the timing and control of the conversation. A semantically correct answer can fail when the agent interrupts the user, misses a correction, treats background speech as an instruction, or pauses long enough that the user repeats or abandons the request.

Evidence should cover turn-taking, barge-in behavior, latency, recovery after interruption, and the final outcome. Uber’s pizza detour shows why transcript correctness is insufficient: the words were recognized, but the agent assigned the wrong authority to them. Review the interaction and resulting action together.

Known failures need evaluation. Unknown failures need discovery.

Pre-release tests and production evaluators cover scenarios the team has already defined. That is essential, but it is not complete. New failure modes appear as models, prompts, tools, users, and environments change.

Use code evaluators, LLM judges, monitors, and alerts for known risks. Use production discovery to find recurring patterns that have not yet become a named metric or test.

Signal in Arize AX continuously reviews production traces, groups recurring and emerging failure patterns, and returns prioritized investigations with supporting evidence and a proposed next step. It can surface issues such as wrong retrieval, invalid tool arguments, skipped steps, loops, or plausible answers produced through a broken trajectory.

Signal is not a reliability score and does not replace a release gate. Instead, it helps a team answer a seemingly simple question: What is failing repeatedly that we did not know to monitor?

Once a pattern is confirmed, make it durable. Add representative traces to a dataset. Define a targeted evaluator where possible. Track the issue in production. The eval suite should become cheaper and more precise as the team learns its own failure modes.

Match the evaluator to the evidence

No single evaluation method covers every reliability failure.

Reliability vs. observability, evaluation, Signal, and the harness

These terms describe different parts of the system.

Agent reliability

The production property: the complete workflow repeatedly keeps the intended outcome true under real conditions.

Agent harness

The runtime and control layer that bounds actions, supplies context, manages state, handles recovery, and decides when work is complete.

Observability and tracing

The evidence layer that records the trajectory, tool calls, retrieval, errors, state transitions, cost, latency, and outcomes.

Evaluation

The judgment layer that decides whether a run, trajectory, session, or outcome met its acceptance criteria.

Signal

The discovery layer that finds recurring or emerging production failures across many traces.

Experiments

The verification layer that compares baseline and candidate behavior before a change ships.

Managed Agents

The investigation and proposal layer that can use approved context, skills, and repositories to produce reviewable work.

Observability does not make the agent reliable. Evaluation does not make the agent reliable. Signal does not make the agent reliable. Each contributes evidence or control to the engineering loop that produces reliability.

The agent harness, tracing, and evaluation guide shows how those parts connect.

How the reliability bar moves when the agent or traffic changes

A reliability budget ages as the product changes. Model swaps alter tool selection. New integrations introduce partial writes. New languages, customers, or longer sessions make old datasets less representative.

Aggregate task success can improve while a high-consequence slice regresses. Keep the baseline, then slice by customer, workflow, language, tool path, run length, environment, and consequence.

TheFork focuses production evaluation on the highest-impact spans and traffic crossing latency or guardrail thresholds instead of scoring every call equally.

Preserve new production failures as regression cases. Lower the priority of checks that no longer represent the product. The outcome stays fixed while the evidence and test distribution change with the agent and its traffic.

How to improve agent reliability

A reliable operating model connects production evidence to the next verified change.

1. Define the task contract and proof of completion

Write down the intended outcome, preserved constraints, allowed tools, permission boundaries, stopping conditions, recovery rules, and evidence that proves the task is complete.

2. Trace the complete trajectory

Capture the request, model decisions, retrieval, tool inputs and outputs, state changes, retries, handoffs, final response, and external proof of completion. Stable task, session, user, prompt, model, and harness identifiers make the evidence comparable across runs.

3. Measure known risks before and after release

Use repeated offline trials, code evaluators, LLM judges, state-based checks, and human review on representative datasets. Run online evals and monitors on production slices where the risk or business impact justifies the cost.

4. Discover failures outside the existing eval suite

Review production traces for recurring and emerging patterns. Signal can automate this discovery step at production scale, attaching evidence and a proposed next step to each issue.

5. Turn confirmed failures into reusable tests

Add representative traces to a regression dataset. Create a deterministic evaluator or narrow LLM judge when the failure is well understood. Use Agent-as-a-Judge when the behavior remains multi-step or context-dependent.

6. Investigate and propose a bounded change

The fix may affect the prompt, context, retrieval, tool contract, routing, state, memory, model, permissions, or orchestration.

Managed Agents can carry an investigation into approved repositories and systems, create evaluation artifacts, and propose reviewable changes. Human engineers should remain responsible for approval and deployment.

7. Verify the complete system

Run the baseline and candidate agent against the same representative tasks and evaluators. Full-agent experiments should compare outcomes, tool use, retrieval, latency, cost, trajectories, and repeated-run variance, not only the final response.

8. Release gradually and watch for recurrence

Use a controlled rollout, keep the rollback path clear, and monitor the specific failure class after release. A fix is not complete when one experiment passes; it is complete when the target issue improves without creating a worse regression elsewhere.

How Arize supports the reliability loop

Arize Phoenix provides an open-source workflow for tracing agents, building evaluations, curating datasets, and running experiments. PXI can help developers inspect traces and carry out interactive AI engineering work inside Phoenix.

Arize AX adds managed production workflows around the same observe, evaluate, and improve loop. Teams can run online evals and monitors, use Signal to discover recurring production issues, use Agent-as-a-Judge for complex trajectory evaluation, investigate with Alyx or Managed Agents, and compare full agent behavior in experiments.

Arize supplies the evidence and workflows. The team still owns the task definition, reliability budget, evaluation policy, and decision about what ships.

Frequently asked questions

What is agent reliability?

Agent reliability is whether the complete agent system repeatedly completes the intended task under real conditions, preserves the user’s constraints, stays within its operating boundaries, and leaves valid evidence that the outcome exists.

A successful API response or fluent final message does not establish reliability when the requested state or artifact is missing.

How is agent reliability different from agent observability?

Agent reliability is the property that the workflow produces the intended outcome.

Agent observability is the capability to inspect the trajectory, tool calls, state changes, errors, cost, latency, and outcomes that show whether the property holds. Observability supplies evidence and helps locate failure; it does not by itself make the agent complete the task correctly.

How is agent reliability different from agent evaluation?

Agent reliability describes how consistently the complete agent succeeds in production.

Agent evaluation is the process used to judge a run, behavior, trajectory, session, or outcome against defined criteria. Evaluation turns evidence into a score or decision. Reliability is the property those judgments estimate over representative tasks, repeated trials, and live traffic.

What role does Signal play in agent reliability?

Signal reviews production traces for recurring and emerging failure patterns, groups related traces into issues, and returns evidence and a proposed next step.

It is useful for discovering failures that the team has not yet encoded in a monitor or evaluator. Signal does not replace task-success metrics, release gates, or human approval.

When should I use Agent-as-a-Judge?

Use Agent-as-a-Judge when the evaluator must explore trace context, reason across several spans or fields, or inspect a multi-step trajectory before scoring behavior.

Use code for deterministic rules and conventional LLM-as-a-Judge for stable semantic criteria with known inputs. Validate important agent-judge results against human-reviewed examples.

Will a better model make my agent reliable?

A better model can improve reliability when model judgment is the limiting factor, but the rest of the workflow can still fail.

Tools can return stale data, permissions can be too broad, retries can duplicate writes, context can drop constraints, and stopping logic can declare victory too early. Measure the model-plus-harness configuration against completed tasks before attributing the result to the model alone.

How do you measure agent reliability?

Measure agent reliability with:

• Task success
• Repeated-run consistency
• Recovery rate
• Robustness under perturbation
• False-completion and no-progress rates
• Severity
• Cost and latency per completed task
• Escalation or abandonment

Grade the trajectory and state left behind, then break results down by risky slices such as workflow, customer, language, tool path, run length, or consequence.

What is a good task success rate?

There is no universal task-success rate.

A useful target is the floor a specific workflow must clear for its users and consequences, measured separately across the riskiest slices. Define the task precisely, set a reliability budget, track high-severity failures separately, and improve from a measured baseline.

How do you improve agent reliability?

Define the task contract, trace the complete workflow, evaluate known risks, discover unknown production failures, turn confirmed issues into regression cases, investigate the likely cause, test the full agent or harness, and release with monitoring and rollback.

Reliability improves when every important production failure becomes durable evidence for the next change.

Inspect what the agent changed

When reviewing the next successful-looking run, ignore the closing sentence for a minute.

Open the trace. Inspect the state that remained. Compare it with the original request, then ask whether the agent could recover if the same run were interrupted halfway through.

That check will tell you more about agent reliability than a clean dashboard or a polished answer.