Tool discovery is how an AI agent finds and loads the tools it needs for a step instead of receiving every tool definition up front. It matters because agents connected to large tool catalogs otherwise spend context on every call and often pick the wrong tool among many similar ones.
For example, suppose an operations agent is connected to 300 tools across ticketing, billing, and infrastructure servers. A user asks it to refund a customer’s last charge. The agent searches the catalog for “refund charge customer,” loads the definitions for two billing tools, and calls them. The other 298 schemas never enter its context.
Build better agents with Arize
Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.
Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.
Why tool discovery matters for agents
Every tool definition an agent receives takes up space in its context window. With a few tools, that cost is small. With hundreds, it adds up on every model call.
A long tool list also makes selection harder. When many tools have similar descriptions, the model is more likely to choose the wrong one.
Tool discovery is useful for:
- Scaling tool catalogs: Connect many servers without sending every schema on every call.
- Improving selection: Give the model a short, relevant list instead of hundreds of near-duplicates.
- Controlling context cost: Spend tokens on the few definitions a step actually needs.
- Adding capabilities mid-task: Let the agent find a tool when a step needs something it has not loaded yet.
Discovery trades a small search step for a much smaller tool list per request. The search step becomes a new place where things can go wrong.
How tool discovery works
Discovery splits tool loading into two stages. First the agent finds candidates. Then it loads full definitions only for the ones it plans to use.
Step 1: Search a lightweight index
A tool catalog typically exposes a lightweight index. It contains tool names, one-line descriptions, and grouping metadata such as a server or category.
The agent searches that index. The search may be keyword-based, embedding-based, or a dedicated tool-search call. It returns a shortlist of candidates.
Step 2: Load the shortlist
The agent requests the full input schemas only for the candidates it intends to use. Those definitions are added to the context for the next model call.
This pattern is often called deferred or dynamic tool loading. Anthropic’s advanced tool use research describes this shape. Exact interfaces vary by provider and framework.
Step 3: Call the tool
Once a definition is loaded, the agent invokes the tool in the usual way. That final step is ordinary tool calling.
Tool discovery with MCP
In a Model Context Protocol (MCP) setup, many servers may be connected while only a few tools matter for a given task. Discovery is what makes that scale workable.
For more on the protocol itself, see Model Context Protocol (MCP). For a catalog-style surface in practice, see Arize’s guide to extending an agent with the Arize AX MCP, CLI, and skills.
Worked example: a refund request
Here is the refund request from the opening, step by step.
1. search_tools("refund charge customer") -> [billing.create_refund, billing.get_charge, tickets.add_note]
2. load definitions for billing.get_charge and billing.create_refund
3. call billing.get_charge(customer_id) to identify the charge
4. call billing.create_refund(charge_id, amount) after confirming the amount
Without discovery, all 300 schemas ride along on both tool calls. With discovery, two schemas do, plus one short search result. The saving grows with catalog size.
The risk is visible in step 1. Suppose the refund tool were described as “issue credit.” The query “refund” might not match it at all, and the agent would never see the right tool.
This means description writing and search evaluation are part of shipping a tool. The wider agent architecture treats context budget as a design constraint, and discovery is one way to manage it.
How to measure tool discovery
Measure discovery separately from tool execution. Build a labeled set of tasks where you know the correct tool for each one.
For each task, record:
- Discovery recall: Did the correct tool appear in the search shortlist?
- Selection: Did the agent then call the correct tool from that shortlist?
- Definition cost: How many tokens did the loaded definitions consume?
- Fallbacks: How often did a missing tool cause a fallback or a wrong-tool call?
Score discovery and selection in Python
The following script scores four hypothetical tasks. It separates tasks where search missed the right tool from tasks where the agent saw the right tool and chose another.
tasks = [
# (correct tool, shortlist returned by search, tool the agent called)
("billing.create_refund", ["billing.create_refund", "billing.get_charge"], "billing.create_refund"),
("billing.create_refund", ["billing.get_charge", "tickets.add_note"], "tickets.add_note"),
("infra.restart_service", ["infra.restart_service", "infra.get_status"], "infra.get_status"),
("tickets.add_note", ["tickets.add_note", "tickets.create"], "tickets.add_note"),
]
found = [correct in shortlist for correct, shortlist, _ in tasks]
chosen = [called == correct for correct, _, called in tasks]
print(f"discovery recall: {sum(found)}/{len(tasks)}")
print(f"selection accuracy when found: "
f"{sum(c for f, c in zip(found, chosen) if f)}/{sum(found)}")
print(f"end-to-end correct: {sum(chosen)}/{len(tasks)}")
for (correct, _, called), f, c in zip(tasks, found, chosen):
label = "ok" if c else ("selection failure" if f else "discovery failure")
print(f"{correct:<24}{label}")
Output:
discovery recall: 3/4
selection accuracy when found: 2/3
end-to-end correct: 2/4
billing.create_refund ok
billing.create_refund discovery failure
infra.restart_service selection failure
tickets.add_note ok
Two of the four tasks failed, but for different reasons. The fix for a discovery failure is a better description or search. The fix for a selection failure is in the prompt, the tool descriptions on the shortlist, or the model.
How to debug tool discovery in traces
In a trace, the two failure types look different.
- Discovery failure: The agent makes a confident call to the wrong tool, or asks an unnecessary clarification question. The correct tool never appeared in the search result.
- Selection failure: The correct tool is in the shortlist, and the agent calls a different one.
The run-level view of a trace is where those two cases separate. Look at the search span’s query and results before looking at the tool call.
Also track catalog drift. Renamed or re-described tools change search behavior without any code change in the agent. This is an agent maintenance task, not a model problem.
For more on tracing wrong-tool calls back to their cause, see debugging agents.
Limitations of tool discovery
Discovery adds a step to every task, and that step can fail. Several problems sit outside what discovery can fix.
- Bad catalogs: Duplicate tools with near-identical descriptions, missing tools, and stale schemas all surface as agent errors. They are actually index errors.
- Exact identifiers: Embedding-based search can miss exact identifiers, error codes, and part numbers that keyword search would catch. Hybrid search or pinned critical tools are common mitigations.
- Permissions: Finding a tool must never imply being allowed to call it. Authorization checks belong at call time.
- Reproducibility: The loaded tool set changes from run to run. Record it so each run can be reproduced and audited.
Tool discovery vs. loading every tool
Loading the full catalog is simpler. There is no search step, no discovery failure, and context cost is predictable. It works well below a few dozen tools.
Discovery wins when the catalog is large or rarely used in full. A common middle ground is to pin frequently used tools in the base context and use discovery for the long tail.
Tool discovery vs. skills
Agent skills differ in scope. A skill packages instructions, and sometimes tools, for a type of task. It is loaded as a bundle.
Tool discovery retrieves individual tool definitions at the moment of need. The two work together, since a loaded skill can still search for a specialized tool. They solve different problems, so keep them separate in design documents.
Frequently asked questions
How many tools do I need before tool discovery is worth it?
There is no fixed threshold. The pressure comes from total schema tokens per call and from selection accuracy as the list grows. If either one hurts, or your catalog spans many servers, discovery is worth its extra step.
Is tool discovery part of MCP?
Not directly. MCP defines how tools are listed and called. Discovery layers such as search, ranking, and deferred loading are built by clients and frameworks on top of that listing. Check what your specific client implements.
What should a tool description contain for good discovery?
Include the action, the object it acts on, and the words a user or agent would naturally use, including common synonyms. Descriptions written only from internal implementation names search poorly.
Can an agent load a tool mid-task that it did not plan for?
Yes, and that is the point of discovery. The agent searches when a step needs a capability it does not have loaded. Record the loaded set for each run so you can reproduce and audit it.
Does tool discovery decide which tools an agent is allowed to use?
No. Discovery only controls which definitions the agent sees. Permission checks should run when the tool is called, regardless of how the agent found it.
Putting tool discovery to work
Start by writing tool descriptions that state the action, the object, and the words users actually say. Pin the handful of tools that most tasks need.
Build a labeled task set and measure discovery recall separately from selection accuracy. Rerun it whenever tools are renamed or re-described.
Trace every search step alongside the tool calls it leads to. Tools such as Arize Phoenix can capture those spans, so you can tell whether a wrong-tool call started with search or with selection.
