A multi-agent system is an arrangement of several agents working on parts of the same task, each with its own instructions, tools, and context. One might plan, another retrieve, another write, another check the result. They exchange work through handoffs, shared state, or messages, and something decides which of them runs when.
The term is older than language models. Distributed AI used it for decades to describe populations of autonomous entities negotiating, competing, or cooperating, and that literature is where words like agent communication and coordination protocol come from. The current usage is narrower and more practical: several LLM-driven agents inside one application, usually with a coordinator.
Keep the neighbors distinct. A multi-agent system is the set of participants. Orchestration is the act of coordinating them. A workflow is the path they follow. The arrangement is what makes it multi-agent, not the fact that coordination is happening, because a single agent needs coordination too.
Key takeaways
- A multi-agent system is defined by the participants: several agents with separate instruction sets, tool access, and context, working toward one outcome.
- Split into multiple agents for a concrete reason such as different tool permissions, separate context budgets, or genuine parallelism. Splitting for tidiness usually costs more than it returns.
- The failure modes that appear only above one agent are handoff context loss, disagreement with no tiebreak, duplicated work, and cascading errors that no single agent sees.
- Cost and latency compound. Each additional agent adds its own model calls, and a supervisor pattern pays for the same context more than once.
- Debugging requires a session-level view. A trace scoped to one agent shows a component that behaved reasonably given bad input.
Common topologies
Supervisor and workers. A coordinating agent decomposes the task and delegates to specialists. The most common shape, and the easiest to reason about. The supervisor becomes the bottleneck and the single point of misjudgment.
Pipeline of specialists. Research feeds drafting feeds editing. Predictable and inspectable. Errors propagate forward, and later stages have no way to know the input was wrong.
Peer review or critic. One agent produces, another evaluates and sends work back. Catches real problems. Also loops forever when the critic is never satisfied, unless you cap the rounds.
Handoff. Control transfers between agents, each fully owning the task while it holds it. This is the pattern behind a lightweight experimental approach to multi-agent coordination that made handoffs the core primitive rather than a special case of delegation.
Shared state. Agents read and write a common workspace rather than messaging each other. Scales to more participants and introduces write conflicts and stale reads, the same way it does in any concurrent system.
The failure modes that need more than one agent
Handoff context loss. The most common one by far. Agent A had a five-turn conversation, then passes agent B a one-paragraph summary. B solves a slightly different problem, competently, and nothing errors. In a trace this looks like two successful runs and one wrong answer.
Disagreement with no tiebreak. A researcher says the record shows one thing, a verifier says another, and nothing in the system resolves it. Whichever agent speaks last wins, which is not a decision rule anyone chose.
Duplicated work. Two agents retrieve the same documents, or both call the same expensive API, because neither can see what the other did. Shows up first in the bill.
Cascading errors. A bad retrieval in step one gets summarized in step two, cited in step three, and defended in step four. Each agent behaved correctly given its input.
Diffuse responsibility for stopping. With one agent, one loop needs a step limit. With five, the run can continue as long as any pair keeps passing work back and forth. Budgets have to apply to the session, not just to each agent.
Attribution. When the output is wrong, which agent was at fault is genuinely hard to answer without spans that carry the handoff payload. This is why tracing and evaluating agents across a whole session matters more here than for a single loop: the interesting information is in what crossed the boundary, not inside either agent.
When multiple agents are worth it
Good reasons to split:
- Different permissions. The agent that can write to production should not be the one parsing untrusted input.
- Separate context budgets. A long research phase would otherwise crowd out the instructions the writing phase needs.
- Real parallelism. Five independent lookups genuinely run at once.
- Different models. A cheap model for triage, a strong one for the hard step.
- Independent evaluation. You want to score retrieval quality separately from synthesis quality, with different criteria.
Weaker reasons: it maps to your org chart, the framework tutorial did it that way, or one prompt was getting long. A single agent with well-designed tools frequently beats a multi-agent version of the same task, with fewer moving parts and one trace to read.
The cost calculation has shifted, though. Falling per-token prices changed the economics of running several models on one task, and designs that were obviously too expensive a year earlier are now merely expensive. Latency has not improved the same way. Sequential agents still add their latencies together, and a supervisor round trip between every step is felt by the user.
FAQ
What is the difference between a multi-agent system and a single agent with many tools?
Participants. In a single-agent system there is one instruction set, one context, and one decision-maker calling functions. In a multi-agent system each participant has its own instructions and its own view of the task, which is what creates handoff and disagreement failures. If your “agents” share one prompt and one message history, you have one agent with tools.
How many agents is too many?
Watch the marginal trace instead of the count. When adding an agent makes runs harder to attribute, adds a handoff that loses information, or introduces a decision no one owns, you have passed the point. Plenty of production systems top out at three.
How do you test a multi-agent system?
At two levels. Test each agent in isolation against fixed inputs, which is ordinary evaluation. Then test the interactions: does the handoff carry what the next agent needs, does the system converge, does it terminate, does it reach the same outcome twice on the same input. The second level is where multi-agent bugs live, and it needs recorded sessions rather than unit tests.
What happens when two agents disagree?
Whatever you designed, and if you designed nothing, it is arbitrary. Options are a deciding agent, a rule that prefers one role, a confidence threshold, escalation to a human, or failing the run. Picking one explicitly is the difference between a system and a conversation.
Do multi-agent systems need a different observability setup?
They need the same primitives applied at a wider scope. Spans per agent are not enough on their own. You want a session that ties every agent’s spans to one task, handoff payloads recorded as data, and evaluations that can score the trajectory rather than only the last message.
