What Is Tool Failure?

Tool failure happens when an external function, API, retriever, database, browser, code runner, or other tool call fails or returns bad data. The agent may have selected the right tool and still fail because the tool timed out, returned an error, or produced unexpected output.

Underneath that definition sit two different problems, and separating them is the whole job. Either the tool erred, or the agent used it wrong. A 504 from a payments API and a date string of next Tuesday sent into a field that requires ISO 8601 land in the same alert. One is fixed with a timeout, a retry policy, and a conversation with the vendor. The other is fixed with a tighter schema and a better tool description. A team tracking a single tool_error_rate will keep applying one fix to the other problem.

Key takeaways

• Tool failures split into the tool erring, the agent using it wrong, and the tool returning a plausible wrong result. Each has a different owner and fix.
• The classification is mechanical: read the tool span’s status and error type, then read the arguments the agent passed.
• Retries help transient tool errors and do nothing for malformed arguments, so retry policy should depend on error class.
• The hardest case to detect is a 200 with an empty or stale payload, because the agent will narrate it as an answer.
• Track tool-side and agent-side failures as separate metrics. A combined error rate hides which team owns the regression.

Class 1: the tool erred

The call was reasonable and the dependency did not hold up its end.

• Timeout. The tool exceeded its budget. The agent may retry, hang, or proceed without the data.
• Server error. A 500 or 503, usually transient, occasionally the start of an incident.
• Rate limit. A 429, a capacity problem rather than a correctness problem, which agents make worse by retrying immediately.
• Auth failure. An expired token or rotated key. Reproduces perfectly and constantly until fixed.
• Malformed response. A 200 whose body does not match the documented shape, so parsing fails downstream of a successful call.
• Schema drift. The dependency renamed a field or changed an enum. Nothing in your repository changed and every call now fails identically.

These are infrastructure and integration problems, and the fixes are the ordinary ones: explicit timeouts, capped retries with backoff and jitter, idempotency keys on anything that writes, a fallback path, and an error class the agent loop can branch on. Most of it belongs in the harness rather than the prompt, which is the point of harness engineering for reliable agents. Retries and budgets are code, and asking a model to be careful is not a substitute.

Class 2: the agent used the tool wrong

The tool behaved exactly as documented. The call was still wrong.

• Malformed arguments. A required field omitted, a type mismatch, an enum value that sounds right and is not in the schema, a natural-language date in a timestamp field.
• Invented identifiers. An order ID in the correct format that does not exist. The tool returns a clean 404 and the agent decides what that means.
• Wrong tool. create_ticket when the task called for get_ticket_status. Common when tool descriptions overlap.
• Right tool, wrong time. Calling submit_order before the address is validated, or searching before rewriting a conversational fragment into a standalone query.
• Missing precondition. Calling a tool that needs a session ID the agent never obtained.
• The call that never happened. The agent answered from memory instead of calling the tool. No failed span exists, which makes it the easiest to miss.

These are model, prompt, and tool-contract problems, and the productive fixes are on the contract side: constrain the schema so invalid values cannot be expressed, state in the description when the tool should not be used, cut the number of near-duplicate tools, validate arguments before execution so the agent gets a specific correction, and give one worked example per tool.

Class 3: the tool succeeded and the result was wrong

The call returned 200, the payload parsed, and the content was stale, empty, or incorrect. A cached balance from before the last payment. An empty list because a filter was wrong. A write endpoint that acknowledged the request and did nothing.

This class reaches users, because every signal you monitor says success. The agent cannot tell an empty result from a true absence, so it reports “there are no matching records” with the same confidence it reports a real finding. Detecting it needs a correctness check on the returned value or an eval comparing the final answer against a source of truth.

Making the call in a trace

Every tool call should be a span carrying its name, arguments, return value, status, latency, and error metadata. With those fields present, the span, trace, and session model turns classification into four questions asked in order:

1. Did the span record an exception or a non-2xx status? The tool erred.
2. Did the dependency return a validation error describing the input? The agent used it wrong.
3. Did the call succeed with a valid result that did not serve the task? Wrong tool or wrong time.
4. Did it succeed with a valid-looking result that was incorrect? Class 3, and the tool or its data is the suspect.

Doing this at volume rather than one session at a time is what an agent observability platform is for: failure rate per tool, per argument, and per error class, so a flaky payments API and a model guessing at enum values show up as two different lines.

FAQ

How do I tell whether the model or the tool failed?

Read the arguments and the status on the same span. Invalid arguments with a validation error means the model failed. Valid arguments with a timeout, 429, or 500 means the tool failed. Valid arguments, a clean response, and a wrong outcome means either the tool returned bad data or the agent called the wrong tool for the task, and the task description is what separates those two.

Should the agent see tool errors?

It should see a controlled version. Passing a raw stack trace or vendor error blob into the prompt invites the model to narrate the error as if it were content, which turns a tool failure into a wrong answer. A short typed message such as RATE_LIMITED, retry later is enough to correct course without inviting improvisation.

How many times should an agent retry a failed tool call?

Retry transient classes only, two or three times, with exponential backoff and a cap. Never retry an argument error, since the same arguments produce the same failure while burning tokens and quota. Anything that changes state needs an idempotency key before a retry is safe at all.

What if the tool returns an empty result?

Treat empty as a distinct case rather than as an answer. Be explicit: if the result set is empty, broaden the query once or say nothing was found, and never assert that a record does not exist based on one filtered query. An empty payload is also worth alerting on when a tool that normally returns rows stops doing so.

Is a retrieval miss a tool failure?

Only when retrieval is invoked as a tool and the call itself broke, such as a vector store timeout. If the retriever ran and returned the wrong passages, that is a retrieval quality problem with its own causes, and filing it under tool failure sends you to inspect infrastructure when the issue is ranking, chunking, or the query.