What Is Agent Debugging?

Debugging agents is the practice of tracing and evaluating an agent’s full execution path to find why it failed. You inspect model calls, prompts, retrieved context, tool inputs and outputs, state changes, retries, policies, and the final response.

The difference from debugging ordinary software is where the bug lives. There is usually no stack trace, no exception, and no line of code that is wrong. The agent made a sequence of decisions, one was bad, and the rest were reasonable given that one. So the work is less about stepping through code and more about reading a transcript of decisions in order and finding the first one that does not follow.

Key takeaways

• Find the first step that went wrong, not the step where the error surfaced. Those are usually different spans.
• Read the trajectory forward. Reading backward from the failure finds steps that look correct given the bad input they were handed.
• If you cannot locate the failing session at all, that is the bug to fix first: surface a session ID with every response.
• Agent runs are not reproducible by replay alone. Recorded tool responses and pinned model versions are what make a repro possible.
• The fix belongs at a specific layer: prompt, tool contract, control flow, or model. Guessing at the layer is how a rate limit gets fixed with prompt edits.

Step 1: find the run

Start from a specific failed session, not a summary. A dashboard telling you correctness dropped four points cannot be debugged. One session where the agent booked the wrong date can be.

This step is where most teams discover their real gap. If a user reports a bad answer and you cannot map that report to a session, no amount of tracing helps. Return a session ID with the response, log it with any feedback event, and make sessions searchable by user, tool name, error status, and eval label.

Step 2: read the trajectory in order

Open the session and read every span from first to last: the system prompt actually sent, the retrieved chunks actually included, each tool call with its arguments and return value, each model response, and the state carried between turns.

Read forward. The instinct is to open the last span, and the last few spans are almost always locally correct because they processed bad input faithfully. Walking forward, ask one question per step: is this output right given this input? The first no is the origin, and everything after it is consequence.

The first divergence usually looks like one of these:

• A tool call whose arguments do not match what the user asked for.
• A retrieval span whose chunks do not contain the answer, followed by an answer anyway.
• A plan that omits a step the request required.
• A tool result the next span restates incorrectly.
• A turn where a constraint from earlier in the session stopped being honored.

Step 3: name the failure before fixing it

Once you have the first bad step, classify it. Wrong tool, malformed arguments, ignored context, no-progress loop, dropped constraint, premature completion. The category matters because it points at a layer:

• Wrong tool or bad arguments points at tool descriptions and schemas.
• Ignored context points at prompt structure and context assembly.
• Loops and premature stops point at control flow and stopping conditions.
• Timeouts, 429s, and 500s point at the harness, not the model.

Most of what you find here is not a model problem. It is missing scaffolding around the model, which is the substance of harness engineering: budgets, retries, validation, and state handling are code you own, and they fix failures no amount of prompt rewriting will.

Step 4: reproduce it

Reproduction is harder here, and it is worth being honest about why. The same prompt to the same model can produce different output across calls. The tools have moved on: the order is shipped, the ticket is closed, the index was rebuilt. And the bug may only appear on turn 14 of a session you cannot easily recreate.

What works in practice is replaying from recorded state:

• Take the exact inputs recorded on the failing span, including the fully assembled prompt.
• Replay against recorded tool responses instead of live tools, so the environment is fixed.
• Pin the model version. A floating alias makes your repro a moving target.
• Change one variable at a time: the tool description, the argument schema, the ordering of context, the model.
• Run the case several times. If it fails four times out of ten, that is the signal, and one passing run does not clear the fix.

All of this depends on having captured the inputs. Every model call, tool call, and retrieval has to land as a span with its inputs and outputs, which can be done in whichever framework the agent already uses rather than requiring a rewrite.

Step 5: keep the case

A fixed bug that is not in a test set comes back. Add the failing session to a regression set with the behavior you expect, so the next prompt change or model upgrade is checked against it. Each entry is a real failure someone already paid for, which makes this set the most valuable artifact debugging produces.

Debugging one session teaches you about one session. Grouping many failing sessions into categories and counting them is a different activity, and it is how you decide which bug is worth a week.

What to capture so debugging is possible at all

• Span per model call with the assembled prompt, response, model version, and token counts.
• Span per tool call with name, arguments, return value, status, latency, and error class.
• Span per retrieval with the query and the chunks actually passed to the model.
• Session and trace identifiers tying multi-turn conversations and subagent runs together.
• Prompt version and configuration on every run.

Without those fields you are guessing whether the model failed, the tool failed, the context was missing, or the loop decided wrong. The agent loop has too many places to hide for guessing to work.

FAQ

How do I debug an agent that only fails in production?

Assume the difference is the input distribution and the environment, not the code. Pull the real session, compare the assembled prompt against what your local run produces, and check the tool responses, since production data has shapes your fixtures do not: empty results, null fields, 400-item lists, unusual locales. Then replay locally with those recorded responses.

Why can’t I reproduce an agent bug?

Three common reasons. Model calls are not deterministic, so a failure that occurs in 30% of runs looks unreproducible if you try once. The environment changed, so tools now return different data than they did during the failure. And session state matters, so a bug that needs 14 turns of accumulated context will not appear in a fresh single-turn run.

Do I need evaluations to debug an agent?

Not for one session, which you can read yourself. You need them to know which sessions to read. Evals on the trajectory flag the sessions worth opening, and they tell you whether a fix generalized or only closed the case you were staring at.

Should I debug with logs or with traces?

Logs tell you that a call happened. A trace tells you where it sat in the sequence, what it received, what it returned, and how long it took, which is what the diagnosis requires. Flat logs from an agent that made 30 nested calls are hard to reassemble into an ordered trajectory, and the ordering is the point.