AI agent tracing and evaluation: The complete developer guide

AI agents don’t fail like ordinary software.

A traditional function usually receives an input, follows a predictable path, and returns an output. An agent may interpret the request, construct a plan, choose among tools, retrieve information, delegate work, recover from errors, and revise its answer before responding.

That creates a difficult class of failure: the final answer can look correct even when the process that produced it was unreliable.

An agent might:

• Call the wrong tool but recover by luck.
• Pass malformed arguments that a forgiving API happens to accept.
• Retrieve the wrong information and then produce a plausible answer.
• Repeat the same action until it reaches a result.
• Skip a required authorization or validation step.
• Give a good answer on one turn while losing the user’s goal across the conversation.

An endpoint-only evaluation sees the request and final response, but doesn’t see what happened between them. That is why agent reliability requires two complementary systems:

• Tracing tells you what the agent did.
• Evaluations tell you whether it did the right thing.

A trace captures the path of a request through model calls, tools, retrieval systems, guardrails, and application logic. An evaluation turns some part of that evidence into a label, score, or explanation. Arize AX describes the same relationship as traces recording what an application did and evals measuring how well it did it.

This guide explains how to instrument an agent, read its traces, design span-, trace-, and session-level evaluations, validate LLM judges, run offline experiments, and monitor agent quality in production. Arize AX is used as a concrete example, but the underlying architecture is based on OpenTelemetry and OpenInference rather than a proprietary tracing format.

TL;DR: Agent tracing records the step-by-step execution of an AI agent, including model calls, tool calls, retrievals, handoffs, errors, latency, and cost. Agent evaluation measures whether those steps and the final outcome met defined criteria. Use tracing to diagnose failures, evaluations to measure quality, datasets to preserve test cases, and experiments to verify improvements.

About the demo: The accompanying video demonstrates the same tracing-and-evaluation workflow using Phoenix, Arize’s self-hostable open source platform. The implementation examples in this guide send OpenTelemetry/OpenInference data to Arize AX. The observability concepts are shared, but exporter configuration, authentication, APIs, and product interfaces differ.

What is AI agent tracing?

AI agent tracing is the practice of recording the complete execution path of an agent run as a structured tree of operations.

In distributed tracing, a trace represents the path taken by a request. The individual units of work inside that trace are called spans. OpenTelemetry uses this model for ordinary software systems, while OpenInference adds AI-specific meanings for operations such as model calls, agents, tools, retrieval, reranking, guardrails, and evaluators.

A trace for a support agent might look like this:

support_agent AGENT

├── classify_request LLM

├── look_up_order TOOL

│ └── database_query CLIENT

├── retrieve_refund_policy RETRIEVER

├── decide_refund_eligibility LLM

├── issue_refund TOOL

└── compose_response LLM

In a trace, each span has a parent and a position in the execution path. In the example above, the hierarchy shows that database_query occurred inside look_up_order, that the refund-policy retrieval informed the eligibility decision, and that issue_refund completed before the agent composed its final response.

This structure lets you determine:

• Which operation triggered another operation.
• Which model decision led to a particular tool call.
• Which retrieved context and tool results were available when a decision was made.
• Whether operations occurred sequentially, in parallel, or as part of a retry.
• Where an error, latency spike, or unexpected cost originated.
• Whether the final response accurately reflected the result of the underlying actions.

Without that hierarchy, developers have to reconstruct the execution path from timestamps and disconnected messages. With it, they can inspect the exact branch where the agent’s behavior diverged from the intended workflow.

A trace also tells you:

• Which operation caused another operation.
• Which model call requested a particular tool.
• Which retrieval result was available when a decision was made.
• Where latency accumulated.
• Which error triggered a retry.
• Which sub-agent performed a piece of work.
• How the final response relates to the preceding steps.

Spans, traces, and sessions

The most useful mental model has three levels:

A session groups together multiple related traces, usually representing an entire conversation or user interaction. Each user turn may produce its own trace, while the session captures the broader context across those turns. This allows you to evaluate behaviors that only emerge over time, such as context retention, consistency, goal completion, and user frustration.

What is an agent evaluation?

An agent evaluation is a repeatable judgment about the quality, correctness, safety, efficiency, or outcome of an agent’s behavior.

An evaluator consumes evidence and returns some combination of:

• A categorical label, such as pass, fail, or needs_review.
• A numeric score.
• A textual explanation.
• A structured failure category.

Evaluations can be produced by human reviewers, deterministic code, an LLM acting as a judge, or a combination of code and model-based checks. They can also be executed inside an agent harness, such as Claude Code, that inspects traces, runs evaluators, tests changes, and compares the results. In Arize AX, reusable evaluators can score spans, traces, sessions, and experiment outputs, while human judgments are captured as annotations and used to validate or improve those automated evaluators.

Tracing and evaluation are not substitutes:

Tracing without evaluation produces more data than a team can review manually. Evaluation without tracing produces scores that often cannot explain why the system failed.

The reliable-agent workflow connects them:

Trace real behavior

↓

Review and label failures

↓

Encode recurring failures as evaluators

↓

Add representative cases to a dataset

↓

Run experiments against proposed changes

↓

Deploy the better version

↓

Run evaluations on production traces

↓

Discover the next failure mode

Why evaluating agents is harder than evaluating a single model call

A single model call can often be evaluated in isolation. Given the system prompt, the user prompt, and the model’s response, you can determine whether the output followed the instructions, answered the request correctly, and remained grounded in the available context.

An agent is different. It performs a sequence of actions before producing its final answer, creating several additional dimensions that also need to be evaluated.

The outcome and the path can disagree

A successful outcome does not prove that the process was good. The agent may have used unnecessary tools, violated a required order of operations, ignored an error, or arrived at the answer through an unstable path.

Conversely, a sensible path can still end in failure because an external tool was unavailable or returned bad data.

You therefore need to ask two separate questions:

1. Did the agent accomplish the task?
2. Did it accomplish the task through an acceptable process?

Trace-level evaluation is especially useful for the second question because it can inspect ordering, repetition, tool use, and recovery across the complete run. It can also evaluate end-to-end task completion and final outcomes when the necessary evidence spans multiple operations. Individual tool calls may look correct in isolation while their ordering, repetition, or combination is nonsensical.

Multiple trajectories may be valid

For many tasks, there is no single correct sequence of tool calls.

A travel agent might retrieve flights before hotels or hotels before flights. Both can be valid. A support agent, however, may be required to verify account ownership before changing a subscription.

This means trajectory evaluation should usually judge behavioral invariants, not exact choreography.

Useful invariants include:

• Required steps occurred.
• Forbidden steps did not occur.
• Preconditions were checked before consequential actions.
• Tool arguments were valid.
• The agent did not repeat work without a reason.
• Tool failures were handled correctly.
• The resulting path was reasonably efficient.

Use exact trajectory matching only when the workflow itself is deterministic or policy-mandated.

Errors compound

A weak retrieval step changes the context passed to the next model call. That decision may cause the wrong tool to run, and the tool result will then influence the final response.

The visible failure may occur near the end of a trace even though the first meaningful divergence happened several spans earlier.

That is why effective debugging asks a key question: Where did this run first become likely to fail?

Many agents are stateful or operate in multi-turn sessions

An answer that looks appropriate in isolation may be wrong in the context of the preceding conversation. Session-level evaluation is needed for criteria such as context retention, consistency, user frustration, goal drift, and resolution across turns.

What should you trace in an AI agent?

Trace enough information to reconstruct the agent’s observable decision process without indiscriminately recording sensitive data.

1. Request and deployment identity

You should look to capture:

• Trace ID and span ID.
• Session ID.
• Model configuration version.
• Agent or workflow name.
• Relevant feature flags.
• An appropriately pseudonymized user or tenant identifier.

These attributes make it possible to compare failures by version, customer segment, environment, prompt, or experiment variant.

2. Model calls

For each model invocation, capture the available non-sensitive subset of:

• Input and output messages.
• Model and provider.
• Invocation parameters.
• Tool definitions advertised to the model.
• Tool calls requested by the model.
• Token usage.
• Cost.
• Latency.
• Finish reason.
• Error status.

OpenInference defines standard attributes for model names, messages, invocation parameters, tools, token counts, and costs.

3. Tool execution

A model asking to use a tool and your application actually executing that tool are different events.

Trace:

• Tool name.
• Tool-call ID.
• Arguments.
• Sanitized result.
• Latency.
• Error.
• Retry count.
• Authorization or approval state.
• Idempotency information for consequential actions.

This distinction is easy to miss. Provider auto-instrumentation may capture the model’s requested function call without capturing the application code that executes it. Arize’s OpenAI tracing guidance recommends wrapping the agent loop and tool execution in manual spans when auto-instrumentation does not cover them.

4. Retrieval and memory

For retrieval or memory operations, capture:

• Query.
• Document or record identifiers.
• Relevance scores.
• Document metadata.
• Selected context.
• Cache behavior.
• Memory reads and writes.
• Data freshness when relevant.

Avoid logging entire documents by default when identifiers and selected excerpts are sufficient.

5. Control flow

Represent the parts of the system that determine what happens next:

• Router decisions.
• Plans.
• Workflow nodes.
• Agent handoffs.
• Reflection or retry decisions.
• Guardrail checks.
• Human approval steps.
• Termination conditions.

You shouldn’t make access to hidden chain-of-thought a prerequisite for observability. Trace externally meaningful decisions, selected actions, structured plans intentionally produced by the application, tool interactions, and concise reasoning summaries when the model or application exposes them for that purpose.

6. Outcomes and operational signals

Capture both semantic and operational outcomes:

• Task completion.
• Final response.
• Error status.
• Turns or steps to resolution.
• Total duration.
• Total model calls.
• Total tool calls.
• Total tokens and cost.
• User feedback.
• Escalation or abandonment.

Use OpenTelemetry for transport and OpenInference for AI semantics

OpenTelemetry provides the vendor-neutral tracing foundation: trace IDs, span IDs, parent-child relationships, context propagation, processors, and exporters.

OpenInference is a semantic-convention standard created and maintained by Arize, layered on top of OpenTelemetry, and it defines what AI-specific spans and attributes mean. (In 2026 Arize proposed donating its OpenInference instrumentation libraries to the OpenTelemetry project to expand OpenTelemetry’s GenAI instrumentation coverage.)

Its span kinds include AGENT, LLM, TOOL, RETRIEVER, RERANKER, CHAIN, GUARDRAIL, EVALUATOR, EMBEDDING, and PROMPT. An OpenInference trace remains a valid OTLP trace, so the instrumentation can be sent to compatible backends rather than tied to one proprietary format.

That separation is valuable:

Your application

↓

OpenInference instrumentation

↓

OpenTelemetry SDK and OTLP

↓

Arize AX, Phoenix, or another compatible destination

The backend-specific portion should primarily be exporter configuration, authentication, and platform workflows (not the semantic meaning of your application).

Auto-instrumentation, manual instrumentation, or both?

There are two ways to add tracing to an AI application. Auto-instrumentation automatically captures telemetry from supported model providers and agent frameworks with minimal code changes. Manual instrumentation is where you explicitly create spans and attach metadata around your own application logic, such as orchestration, tool execution, routing, or custom retrieval. Most production systems use a combination of both.

The hybrid approach is the best default for most production systems: let auto-instrumentation capture standard provider and framework events, then add manual spans wherever your application makes important decisions or executes custom logic.

Arize’s documentation describes this as the common production pattern: auto-instrumentation handles provider calls while manual spans cover custom parts of the application, and both join into the same trace through the shared tracer provider.

Arize Skills can guide compatible coding agents through instrumentation and trace-analysis workflows.

Arize AX example: instrumenting an agent with OpenInference

The following example uses Arize AX as the trace destination. The underlying spans use OpenTelemetry and OpenInference conventions.

The same pattern applies to other providers and frameworks. Use the instrumentor that matches your model provider, such as Anthropic or Amazon Bedrock. If your application uses a framework such as LangChain, LlamaIndex, or CrewAI, add the corresponding framework instrumentor as well. In most cases, you should instrument both the framework and the underlying model provider while keeping the register(...) and .instrument(...) flow consistent.

Install the AX exporter convenience package and the instrumentor for your model provider:

pip install arize-otel openinference-instrumentation \

openinference-instrumentation-openai

Configure tracing before the application makes model calls:

import json

import os

from typing import Any
from arize.otel import register

from openinference.instrumentation import (

using_metadata,

using_session,

using_user,

)

from openinference.instrumentation.openai import OpenAIInstrumentor

from opentelemetry import trace

from opentelemetry.trace import Status, StatusCode
tracer_provider = register(

space_id=os.environ[“ARIZE_SPACE_ID”],

api_key=os.environ[“ARIZE_API_KEY”],

project_name=”support-agent”,

)
OpenAIInstrumentor().instrument(tracer_provider=tracer_provider)
tracer = tracer_provider.get_tracer(“support-agent”)

The AX setup registers an OpenTelemetry tracer provider, while the OpenInference instrumentor automatically captures supported model calls. Arize’s current setup follows this same register(...) plus .instrument(...) pattern.

Add decorator-based spans for the custom agent loop and tools

Use OpenInference decorators when a function maps cleanly to one agent, tool, or chain span. They reduce manual attribute handling while preserving the parent-child relationships in the trace.

First, wrap the OpenTelemetry tracer returned by register():

import os

from collections.abc import Callable

from typing import Any
from openinference.instrumentation import (

OITracer,

TraceConfig,

using_attributes,

)
tracer = OITracer(

tracer_provider.get_tracer(__name__),

config=TraceConfig(),

)

When you control the tool definition directly, the equivalent decorator syntax is even simpler:

TOOL_REGISTRY: dict[str, Callable[…, Any]] = {

“look_up_order”: tracer.tool(

name=”look_up_order”,

)(look_up_order),

“issue_refund”: tracer.tool(

name=”issue_refund”,

)(issue_refund),

}
def execute_tool(

name: str,

arguments: dict[str, Any],

) -> Any:

“””Dispatch a tool call to its instrumented implementation.”””

try:

tool = TOOL_REGISTRY[name]

except KeyError as exc:

raise ValueError(f”Unknown tool: {name}”) from exc
return tool(**arguments)

The outer wrapper is intentional. It activates the session, user, and metadata context before the decorated agent function opens its span, allowing those attributes to propagate to the agent and its child spans.

The resulting trace should contain:

support-agent AGENT

├── model request LLM ← automatic

├── tool:look_up_order TOOL ← manual

├── model request LLM ← automatic

├── tool:issue_refund TOOL ← manual

└── model request LLM ← automatic

When instrumenting an agent framework and its underlying model provider, instrument both when the relevant integrations recommend it. The framework instrumentor may capture workflow structure, while the provider instrumentor captures model-specific data.

Protect sensitive data before it becomes telemetry

Prompts, tool arguments, retrieved records, and model outputs can contain personal data, secrets, credentials, or regulated information.

You should never treat the observability backend as the first place where redaction happens. Instead, you should apply data minimization before export:

• Never log API keys, authorization headers, passwords, or session secrets.
• Replace account identifiers with internal pseudonymous IDs.
• Prefer record IDs and approved excerpts over full database objects.
• Redact sensitive tool arguments.
• Separate operational metadata from message content.
• Define retention and access policies.
• Sample or suppress particularly sensitive workflows.
• Test redaction with representative payloads.

OpenInference supports configuration controls for hiding invocation parameters, tool definitions, inputs, outputs, and message bodies.

A useful rule is to trace the information required to explain and evaluate behavior instead of every byte the application handled.

How to debug an agent from its trace

A productive trace review should follow causality rather than simply scanning red spans.

1. Confirm the expected trace shape

Before inspecting the failure, ask what a healthy run should contain.

For a refund request:

verify_identity

→ look_up_order

→ retrieve_policy

→ determine_eligibility

→ issue_refund or explain_denial

Missing spans can be as important as failing spans.

2. Find the first meaningful divergence

Start at the root and move forward until actual behavior differs from expected behavior.

The first divergence might be:

• A router choosing the wrong branch.
• A retrieval returning irrelevant evidence.
• The model omitting a required tool call.
• Invalid tool arguments.
• A tool error the agent ignores.
• A repeated step indicating a loop.
• A handoff that loses necessary context.

Later failures are often symptoms of this earlier decision.

3. Inspect the evidence available at that moment

For the span where behavior diverged, inspect:

• What input did the component receive?
• Which instructions and tool definitions were visible?
• What retrieval results were available?
• What application state was present?
• Did earlier data become truncated or transformed?
• Did the component receive stale or contradictory context?

This separates model failure from context-construction failure.

4. Compare against successful traces

One failing trace is useful. A failing trace next to a successful trace for a similar request is much more useful.

To start, try comparing the following:

• Tool sequence.
• Tool arguments.
• Retrieved data.
• Prompt or model version.
• Step count.
• Retries.
• Token consumption.
• Latency.
• Final answer.

5. Turn the finding into a reusable failure category

You should never stop at the point where you realize “this trace failed.”

Label the mechanism:

wrong_tool

missing_required_step

invalid_tool_arguments

irrelevant_retrieval

ignored_tool_error

unnecessary_loop

unsupported_claim

lost_session_context

policy_violation

Those labels become the foundation of your evaluation system.

The four dimensions of agent evaluation

A complete evaluation design makes four choices:

1. What scope does the evaluator inspect?
2. Who or what performs the judgment?
3. When does the evaluation run?
4. How are scores interpreted and aggregated?

An evaluator may return a pass/fail label, a category, or a continuous score. Deterministic checks such as schema validation may behave like unit tests, while semantic evaluators and human reviewers involve judgment and can disagree.

For each evaluator, define:

• the threshold that counts as passing;
• the acceptable pass rate across a representative dataset;
• any critical failures that must always block a release; and
• how the evaluator will be calibrated against human-reviewed examples.

A release should usually depend on aggregate performance and critical-case failures rather than requiring every example to pass. For example, a helpfulness evaluator might need to pass 90% of cases, while a permission or safety check may require 100%.

This is more technically accurate than saying all evals are inherently non-binary. Arize’s own guidance emphasizes fixed labels, calibration against human review, and checking whether an evaluator’s judgments agree with the decisions people actually care about.

Evaluation scope

Use the narrowest scope capable of answering the question. Wider scopes increase judge context, cost, and noise. A production application will usually use several evaluators at different levels rather than one evaluator that attempts to judge everything.

Evaluation method

Pro tip: Prioritize code whenever the criterion is genuinely deterministic. Use an LLM judge for semantic questions that can be answered from the provided context. When evaluation requires gathering evidence, using tools, or investigating a multi-step trajectory, use an agent as a judge. Continue using human review to discover failure modes, resolve ambiguous cases, and calibrate automated evaluators.

Evaluation timing

Offline evaluation runs against a curated dataset or exported traces. Online evaluation continuously or periodically scores production traces, often with filters and sampling.

Arize AX supports running evaluation tasks over production trace data, while datasets and experiments provide the repeatable offline comparison workflow.

Evaluation thresholds and pass rates

An evaluator can return a binary label, a categorical result, or a continuous score. That output still needs an interpretation rule. A pass label from one example does not automatically mean the system is ready to ship, and a single failed example does not always mean the release should be blocked.

Define thresholds at two levels:

1. Example-level threshold: What score or label counts as passing for one evaluated run?
2. Dataset-level pass rate: What percentage of representative examples must pass before the system meets the release criterion?

For example, a groundedness evaluator might classify each response as grounded or ungrounded, while the release gate requires at least 95% of the evaluation dataset to be grounded. A task-completion evaluator might produce a score from 0 to 1, with individual runs passing at 0.8 or higher and the overall dataset requiring a 90% pass rate.

Different evaluators should have different tolerances. A helpfulness evaluator may allow occasional disagreement because the criterion involves judgment. A deterministic check for permissions, required approvals, or prohibited actions may require a 100% pass rate. Critical cases should be identified separately so that one severe failure can block a release even when the aggregate pass rate remains high.

Semantic evaluators also make mistakes. Calibrate judges against human-reviewed examples, measure how often they agree with reviewers, and inspect false positives and false negatives before using their scores as release gates. Revisit thresholds when the rubric, judge model, application behavior, or production distribution changes.

A useful release policy records:

• the evaluator version and rubric;
• the example-level passing rule;
• the required dataset-level pass rate;
• any critical cases that must always pass; and
• the baseline or previous version used for comparison.

The goal is to make the decision rule explicit. Teams should know what each score means, how much evaluator error is acceptable, and what evidence is required before a change moves into production.

The agent evaluation matrix

A mature agent rarely has one “quality score.” It has a set of targeted measurements.

These measurements should remain separate. A single blended score can hide the difference between a correct but expensive agent and a fast agent that routinely selects the wrong tools.

Evaluate the outcome and the process separately

A strong evaluation suite contains at least two families of checks.

Outcome evaluations

Outcome evaluations ask whether the user’s goal was satisfied.

Examples include:

• Was the package status reported correctly?
• Was the requested report generated?
• Did the environment reach the expected final state?
• Was the final answer supported by the available evidence?
• Did the conversation resolve the issue?

Whenever possible, evaluate the environment state, not just the agent’s claim.

For example, an agent saying “Your refund has been issued” is not proof that the refund tool succeeded. The authoritative outcome is the payment system’s state.

Process evaluations

Process evaluations ask whether the agent behaved acceptably while reaching the result.

Examples here include the following:

• Did it call the appropriate tool?
• Did it pass the correct arguments?
• Did it perform authorization before taking action?
• Did it use retrieved evidence correctly?
• Did it recover from a transient failure?
• Did it avoid unnecessary loops?
• Did it delegate to the right sub-agent?
• Did it terminate at the right time?

The outcome and process should be visible as separate dimensions:

Outcome pass + Process pass

→ Ideal run
Outcome pass + Process fail

→ Lucky or unsafe success
Outcome fail + Process pass

→ Sound behavior blocked by data, tooling, or environment
Outcome fail + Process fail

→ Agent or workflow failure

The second and third categories are especially valuable because an endpoint-only evaluation tends to collapse them into misleading results.

How to build an agent failure taxonomy

Start with real traces instead of a generic list of evaluation metrics.

To start, you should consider reviewing:

• User complaints.
• Low-rated sessions.
• Tool errors.
• Long or expensive traces.
• Abandoned conversations.
• Safety escalations.
• Runs where humans overrode the agent.
• Inputs that succeeded inconsistently.

For each example, identify the first meaningful failure mechanism.

A support-agent taxonomy might look like this:

This is more actionable than starting with a generic “helpfulness evaluator.” Each category identifies what failed, where it can be observed, and what kind of evaluator can detect it.

How to write a reliable LLM-as-a-judge evaluator

An LLM judge should be treated as a measurement instrument.

A strong evaluator prompt includes:

• One clearly defined criterion.
• The exact evidence the judge may use.
• Explicit pass and fail conditions.
• Boundary cases.
• A structured output schema.
• A concise explanation requirement.
• Instructions for insufficient evidence.
• Examples representing difficult distinctions.

Avoid asking one judge to simultaneously score correctness, tone, safety, efficiency, groundedness, and tool use. Separate evaluators produce more interpretable results.

Example: trajectory-quality rubric

You are evaluating the execution trajectory of an AI agent.
USER GOAL

{user_input}
AVAILABLE TOOLS

{tool_definitions}
ORDERED AGENT STEPS

{trajectory}
FINAL RESULT

{final_output}
Evaluate whether the trajectory was safe, logically coherent, and

reasonably efficient.
A trajectory passes when:
1. The selected tools are appropriate for the user’s goal.

2. Tool arguments are consistent with the request and preceding evidence.

3. Required prerequisites occur before consequential actions.

4. The agent does not perform unnecessary repeated work.

5. Tool errors are handled or clearly reported.

6. The final result follows from the observed tool results.
Do not fail a trajectory merely because it differs from a reference path.

Alternative paths are acceptable when they are safe, correct, and no less

effective.
Return a structured result with:
– label: “pass”, “fail”, or “insufficient_evidence”

– score: number from 0.0 to 1.0

– failure_mode: concise category or null

– explanation: one concise paragraph

Structured judgments are preferable to parsing free-form prose. Phoenix evaluation tooling, which can also be used with Arize AX workflows, supports structured evaluator outputs through tool calling.

Continuously validate the evaluator

The evaluator needs its own evaluation loop.

Use the evaluator to measure the agent. Then use human-labeled examples from the agent’s real behavior to measure and improve the evaluator. Repeat this process as the product, user population, failure modes, evaluator prompt, or judge model changes.

Build a human-labeled calibration set

Start with examples that represent the behavior the evaluator will encounter, including:

• Clear passes.
• Clear failures.
• Borderline cases.
• Every known failure category.
• Different user and task segments.
• Different agent versions.
• Adversarial or unusually phrased inputs.
• Cases where the final answer is good but the process is bad.
• Cases where the process is reasonable but an external system fails.

Have reviewers label each example without seeing the automated evaluator’s score. Record disagreements between reviewers rather than forcing immediate consensus, since those disagreements may reveal ambiguity in the product requirement or rubric.

Compare the evaluator with human judgment

Run the evaluator against the labeled set and measure more than average agreement. Depending on the use case, inspect:

• Precision on failures.
• Recall on failures.
• False-positive rate.
• False-negative rate.
• Agreement by failure category.
• Agreement by customer or task segment.
• Stability across repeated judge calls.
• Changes when the evaluator prompt or judge model changes.

The importance of each measure depends on how the evaluator is used. For a production alert, excessive false positives can create alert fatigue. For a release gate on a high-risk workflow, false negatives may allow serious failures to reach production.

Review disagreements

When the evaluator and human reviewers disagree, inspect the example rather than automatically assuming that either side is correct. Disagreement may indicate that:

• The rubric is ambiguous.
• Reviewers disagree about the product requirement.
• The evaluator lacks necessary context.
• The label set is too coarse.
• Several failure modes were combined into one evaluator.
• The judge model is not capable enough for the task.
• The example itself is underspecified.

Use these findings to revise the rubric, evidence provided to the evaluator, label definitions, or evaluator design.

Continue validating with production data

A static calibration set will become less representative as the agent changes. Sample production traces regularly, prioritize new and disputed failure modes for human review, and add those labeled examples to the calibration set.

Revalidate the evaluator whenever you change:

• The agent or its harness.
• The evaluator prompt or rubric.
• The judge model.
• The tools or context available to the agent.
• The user population or task distribution.
• The thresholds used for alerts or release gates.

Track evaluator versions so that changes in product quality can be distinguished from changes in how quality is measured.

The goal is a continuous cycle: evaluators identify product failures, humans label representative product behavior, and those labels are used to validate and improve the evaluators. Over time, this creates a more stable operational definition of acceptable agent behavior.

How to evaluate an agent trajectory

A trajectory evaluation scores the ordered sequence of actions taken during a trace.

A useful trajectory representation includes:

[

{

“step”: 1,

“type”: “tool”,

“name”: “look_up_order”,

“arguments”: {“order_id”: “1847”},

“status”: “ok”

},

{

“step”: 2,

“type”: “retrieval”,

“name”: “refund_policy”,

“documents”: [“policy-returns-v4”]

},

{

“step”: 3,

“type”: “tool”,

“name”: “issue_refund”,

“arguments”: {“order_id”: “1847”, “amount”: 49.00},

“status”: “ok”

}

]

You should avoid sending an unstructured dump of every span attribute to the judge. Instead, try reconstructing the smallest representation that preserves the evidence required by the rubric.

Four trajectory-evaluation strategies

For consequential workflows, combine deterministic invariants with a semantic judge:

Deterministic:

– verify_identity occurred

– verify_identity preceded issue_refund

– issue_refund occurred no more than once
Semantic:

– Was the refund decision supported by the order and policy evidence?

– Was the overall path necessary and efficient?

An example with Arize AX

A trace-level workflow in Arize AX can:

1. Export spans.
2. Group spans by trace.
3. Extract ordered tool calls and their arguments.
4. Run a Phoenix Evals classifier or another evaluator.
5. Attach the result to the root span using the span identifier.
6. Filter, compare, and monitor the resulting evaluation columns.

Arize’s trajectory-evaluation example follows this pattern: group tool-calling spans, classify the ordered trajectory, and log the evaluation to the root span. Its offline evaluation guidance uses context.span_id as the join key between a score and the original trace data.

An abbreviated classifier looks like this:

from phoenix.evals import OpenAIModel, llm_classify
results = llm_classify(

dataframe=trace_level_dataframe,

template=TRAJECTORY_RUBRIC,

model=OpenAIModel(

model=”your-judge-model”,

temperature=0.0,

),

rails=[“pass”, “fail”, “insufficient_evidence”],

provide_explanation=True,

)

The important design work happens before this function call:

• Selecting the right traces.
• Reconstructing the trajectory correctly.
• Supplying only relevant evidence.
• Defining acceptable path variation.
• Validating the rubric against human labels.

Build datasets before and after release

Evaluation datasets should exist before an application reaches production and improve as the application encounters real users.

Before release, build from intended behavior

For a pre-release application, start with a clear definition of what the agent should and should not do. Build curated and synthetic examples from product requirements, business workflows, tool contracts, safety policies, and anticipated user behavior. Pre-production evaluation commonly uses curated, synthetic, and human-annotated datasets to validate an application before release.

Include:

• Representative common requests.
• Important business workflows.
• Expected successful outcomes.
• Expected refusals and escalations.
• Required and forbidden actions.
• Required and forbidden tool calls.
• Ordering and approval constraints.
• Permission and policy boundaries.
• Rare but high-impact edge cases.
• Adversarial or unusually phrased inputs.
• Different user, customer, and environment segments.
• External system failures, timeouts, and incomplete results.

Synthetic examples should be generated from an explicit task and failure taxonomy rather than from arbitrary prompts. Review them for realism, coverage, and alignment with the product requirements they are intended to test.

After release, learn from real behavior

Once the application is in production, supplement the initial dataset with evidence from real traces and user interactions. A random sample of traffic may help measure overall performance, but a useful regression dataset should deliberately include:

• Known production failures.
• Previously fixed regressions.
• New failure categories.
• Rare but high-impact cases.
• Segment-specific failures.
• Cases where the answer is good but the process is bad.
• Cases where the process is reasonable but an external dependency fails.
• Disagreements between automated evaluators and human reviewers.

Production failures can also be used to generate targeted variations. For example, if an agent fails on one unusual refund request, create related examples that vary the wording, customer state, order status, permissions, and tool responses.

Store the expected behavior, not only the expected answer

Each dataset row can contain the input, expected outcome, workflow constraints, environment state, and metadata needed to evaluate the run:

{

“input”: {

“user_request”: “Refund order 1847”

},

“expected”: {

“task_outcome”: “refund_issued”,

“required_tools”: [

“look_up_order”,

“issue_refund”

],

“forbidden_tools”: [

“delete_account”

],

“ordering_constraints”: [

[“look_up_order”, “issue_refund”]

]

},

“metadata”: {

“category”: “refund”,

“risk”: “high”,

“source”: “synthetic_pre_release”,

“requirement_id”: “REFUND-03”

}

}

The source field might later be updated to values such as production_failure, human_review, customer_report, or regression. This makes it possible to compare performance across synthetic cases and real production behavior.

Datasets should evolve throughout the product lifecycle. Before release, they encode the application’s intended behavior. After release, they capture the ways real behavior differs from that intent. Phoenix and Arize AX can then use those examples in repeatable experiments.

A practical rule: every important product requirement should have at least one dataset example before release. Every important production failure should either become a new example or improve an evaluator. Usually, it should do both.

Use experiments to answer “should we ship this change?”

An experiment runs a candidate version of the agent against a fixed dataset and applies the same evaluations to every result.

Candidate changes might include:

• A new system prompt.
• Different tool descriptions.
• A model change.
• A new retrieval strategy.
• Different routing logic.
• A retry policy.
• A redesigned memory system.
• A guardrail.
• A changed tool schema.
• A new sub-agent.

Compare the candidate against the current baseline on:

• Task-success rate.
• Each major failure category.
• High-risk slices.
• Cost.
• Latency.
• Tool error rate.
• Step count.
• Human preference where needed.

Don’t gate releases only on the aggregate average.

A change can improve the overall score while regressing an important minority segment. For example:

Overall task success: +3.2%

Refund workflow success: -8.4%

Average cost: +19.0%

Unsafe-action rate: unchanged

The correct release decision depends on the product’s priorities and risk tolerances, not the headline average.

Arize AX experiments store runs independently so teams can compare models, prompts, and application changes against the same curated dataset. Its CI/CD guidance also supports incorporating experiments into automated release workflows.

Run online evaluations on production traces

Offline evaluation answers whether a known change performs well on known examples. Production reveals behavior that the offline dataset did not anticipate.

Production introduces:

• New users who express familiar goals in unfamiliar ways.
• Existing users who change how they prompt as they learn the system.
• New tool failures.
• Data drift.
• Model-provider changes.
• Unexpected interaction combinations.
• Longer conversations.
• New adversarial inputs.
• Changes in downstream systems.

User behavior is especially important because it rarely remains static. New users may provide detailed instructions, while experienced users often shorten prompts, omit context they expect the agent to remember, and attempt more complex workflows. Different customers and user groups may also develop distinct prompting patterns. Online evaluation should therefore measure performance across user cohorts and over time, rather than treating production traffic as one stable population.

Online evaluation applies validated evaluators to incoming or recently collected traces.

A sensible production strategy combines:

• Deterministic checks on all eligible traces.
• LLM judges on a representative sample.
• Higher sampling for new releases.
• Full or elevated coverage for high-risk workflows where feasible.
• Automatic evaluation of errors, unusually long traces, and expensive outliers.
• Human queues for ambiguous or consequential failures.
• Regular review of emerging prompt and usage patterns.
• Promotion of important production failures into regression datasets.

Segment results by:

• Application version.
• Prompt version.
• Model.
• Tool.
• Workflow.
• New versus experienced users.
• User or tenant cohort.
• Prompt length or interaction pattern.
• Language.
• Risk level.
• Trace duration.
• Session length.
• Token or cost band.

An aggregate failure rate may look stable while one version, tool, workflow, or user cohort is deteriorating. Cohort-level analysis also helps distinguish a product regression from a change in how people are using the system.

When a new usage pattern appears, decide whether the product should support it. Supported behavior should become part of the evaluation dataset. Unsupported or unsafe behavior should inform clearer product constraints, guardrails, or user guidance.

In Arize AX, an online evaluation task connects an evaluator to trace data and defines its filters and sampling. Results can then be analyzed in the context of the production traces they scored.

Evaluating multi-agent systems

A multi-agent system adds delegation and coordination failures to the existing agent failure modes.

Trace each agent or meaningful workflow node as an AGENT span, and preserve relationships across:

• Supervisor-to-worker delegation.
• Agent handoffs.
• Parallel work.
• Shared memory.
• Tool use.
• Aggregation or synthesis.
• Termination.

Useful multi-agent evaluations include:

For custom workflows, Arize AX can use graph node and parent metadata to construct higher-level agent visualizations in addition to the detailed span tree.

Don’t let the visualization replace the underlying trace. A graph is useful for understanding logical flow; the trace remains necessary for inspecting exact calls, attributes, timing, and errors.

Common tracing and evaluation mistakes

A production readiness checklist

Before calling an agent observable and evaluable, verify that:

• Each user turn or agent run produces a complete trace.
• LLM calls, retrievals, tools, guardrails, and custom logic are represented.
• Model-requested tool calls are distinguishable from application-side tool-execution spans, and the two can be correlated using the tool-call ID where available.
• Parent-child relationships reconstruct the real control flow.
• Multi-turn conversations share a session ID.
• Prompt, application, model, and workflow versions are recorded.
• Errors, retries, token usage, cost, and latency are visible.
• Sensitive inputs and outputs are redacted or suppressed appropriately.
• The team has a written failure taxonomy based on real traces.
• Deterministic rules use code rather than an LLM judge.
• Semantic evaluators have explicit rubrics and structured outputs.
• LLM judges are calibrated against human-labeled examples.
• Outcome and process quality are scored separately.
• Important production failures are added to a versioned dataset.
• Candidate changes run against the same dataset before release.
• Release decisions consider critical slices, cost, and latency—not only averages.
• Production traces receive continuous or sampled online evaluation.
• Evaluation failures link directly back to the originating trace.
• Evaluator versions and application versions can be compared over time.

Arize AX and Phoenix

The concepts in this guide apply to either Arize product, but their setup and operating models differ.

Arize AX is the managed AI engineering platform and is the primary example used here. Phoenix is Arize’s open-source observability and evaluation platform, which can run locally or be self-hosted. Both are built around OpenTelemetry and OpenInference.

Take this with you

Reliable agents are not created by adding a dashboard after deployment.

They are created through a connected engineering loop:

1. Trace the agent’s observable behavior.
2. Inspect failures in their causal context.
3. Classify the underlying failure modes.
4. Evaluate outcomes, components, trajectories, and sessions.
5. Validate evaluators against human judgment.
6. Preserve important cases in a dataset.
7. Experiment on prompts, models, tools, and workflows.
8. Monitor production traces with online evaluations.
9. Feed new failures back into the system.

When those pieces share the same evidence and failure taxonomy, agent improvement stops being guesswork.

Trace and evaluate your AI agents with Arize AX, or begin with open-source observability and evals in Phoenix. Arize AX supports structured tracing across spans and sessions as well as online evaluations over trace data.

Frequently asked questions

What is AI agent tracing?

AI agent tracing records the complete execution of an agent run as a hierarchy of spans. It can include model calls, tool requests, actual tool execution, retrievals, handoffs, guardrails, errors, latency, token usage, cost, and the final result.

What is the difference between tracing and evaluation?

Tracing captures evidence about what happened. Evaluation applies criteria to that evidence and returns a judgment such as a label, score, explanation, or failure category. Tracing is primarily diagnostic; evaluation makes quality measurable at scale.

What is a trace-level evaluation?

A trace-level evaluation inspects the complete path of one agent request rather than one isolated operation. It is useful for judging tool ordering, trajectory efficiency, required steps, retries, error recovery, and the relationship between intermediate evidence and the final result.

What is a trajectory evaluation?

A trajectory evaluation measures the quality of the sequence of actions an agent took. It may check exact steps, partial ordering, required or forbidden actions, tool arguments, efficiency, recovery, and whether the path logically supports the outcome.

Should every agent have a golden trajectory?

No. Some workflows have one required sequence, but many agent tasks permit several valid paths. In those cases, define required invariants, forbidden behavior, ordering constraints, and outcome criteria rather than one exact trajectory.

What should be evaluated at the span level?

Use span-level evaluation when all required evidence exists inside one operation. Typical examples include tool selection, tool-argument validity, retrieval relevance, model-response correctness, and guardrail decisions.

What should be evaluated at the session level?

Use session-level evaluation for criteria that require multiple turns, such as context retention, conversation coherence, goal drift, consistency, user frustration, escalation, and eventual resolution.

Are LLM-as-a-judge evaluations reliable?

They can be useful and scalable, but they are probabilistic measurement systems rather than perfect unit tests. Validate the judge against a representative human-labeled dataset, then measure false positives, false negatives, and agreement across failure categories and user or task segments.

Define two separate thresholds:

• Evaluator agreement rate: How often does the judge agree with trusted human labels?
• Application pass rate: What percentage of evaluated examples must meet the quality bar?

Keep these rates separate. Evaluator agreement tells you whether the judge is trustworthy. Application pass rate tells you whether the product is performing well enough.

For a subjective criterion such as helpfulness or trajectory quality, a release might require 90–95% of representative examples to pass rather than expecting every example to pass. Deterministic checks for permissions, required approvals, or critical safety failures may still require 100%.

Use focused evaluation criteria, structured labels, representative examples, disagreement analysis, versioning, and slice-level performance checks. Revalidate the evaluator whenever the judge model, evaluation prompt, application behavior, or production traffic changes.

This distinction fits the broader framework: evaluators may return labels or numeric scores, but those outputs need an explicit interpretation rule and calibration against human judgment.

Should an evaluator use the same model as the agent?

It can, but a different judge model may reduce correlated errors and provide an independent perspective. The more important requirement is demonstrated agreement with the human judgments that define product quality.

What is the difference between OpenTelemetry and OpenInference?

OpenTelemetry provides the general tracing infrastructure and protocol. OpenInference adds AI-specific semantic conventions for operations such as agents, model calls, tools, retrieval, reranking, guardrails, prompts, and evaluations.

What is the difference between offline and online agent evaluation?

Offline evaluation runs a candidate agent against a repeatable dataset to decide whether a change should ship. Online evaluation scores sampled or filtered production traces to detect regressions, new failure modes, and behavior that was not represented in the test dataset.

Can tracing expose sensitive data?

Yes. Prompts, completions, retrievals, and tool arguments may contain sensitive information. Apply redaction, suppression, access controls, pseudonymization, retention policies, and data minimization before exporting telemetry.