- Multi-agent orchestration is the control logic that divides a task among agents, governs their handoffs and recovery, and decides when evidence is sufficient to close the work. Use it only when separate permissions, context boundaries, independent work, or independent verification improve the task beyond what one agent can safely complete.
- A single agent is the baseline. Add coordination only when the task has real permission, context, parallelism, or verification boundaries that one agent should not own alone.
- Evaluate the completed task and the route through it. A fluent final response or a successful tool call is not enough when the task changes an external system.
- Treat a coordinator change as an engineering change. Compare it with a baseline on the same cases, keep the failures, and turn confirmed coordination mistakes into regression coverage.
A policy change blocks invoice exports for a customer group. Your investigation agent finds the cause, another agent prepares the patch, and the deployment tool reports success. The final response says the incident is resolved, even though the affected customers still cannot export.
The implementation agent may have received the wrong tenant or an old policy version, while the verifier may have accepted the deployment response without checking production. To locate the failure, follow the evidence each role received and the decision that allowed work to continue.
What is multi-agent orchestration?
Multi-agent orchestration is the logic that decomposes a task, assigns work across agents, preserves task state and artifacts, handles recovery, and decides when the work is complete. Within a multi-agent system (e.g., the full set of agents, models, tools, and memory) orchestration is the control layer: it governs how the workers cooperate, in what order, and under what completion criteria.
For the export incident, your system might use an investigator with read-only access to customer state, an implementation agent that can edit the repository, and a verifier that can inspect production. Although these roles may share a model, they use different instructions and permissions, so the orchestrator must decide when each can start and what it needs to receive.
The orchestrator sends the incident from investigation to implementation and then to independent verification. State, retries, tracing, and evaluation make each step visible and recoverable.
Each agent’s harness supplies its context, tools, permissions, and local execution loop. The orchestrator decides the workflow and handoffs, while the runtime provides checkpoints, queues, and recovery capabilities. Tracing and evaluation record what happened and assess whether the task and its constraints were satisfied.
Multi-agent orchestration vs. a single agent: when separate agents improve the workflow
A support assistant that retrieves an invoice and explains a charge can usually complete the request with one agent. Splitting retrieval and explanation creates a handoff, so you need a reason for that boundary beyond giving the workers different role names.
Independent research creates a different opportunity. In Anthropic’s multi-agent research system, workers pursue separate search directions and return findings to a lead agent. This division helps because each branch can make progress on its own; tightly coupled workers instead spend more effort synchronizing context and resolving conflicts.
Choose the pattern from the work itself. Pass a result forward when the next step needs it; run branches in parallel only when they can work independently; route only when the request clearly requires a specialist. These are the three core multi-agent orchestration patterns — sequential, parallel, and routed (handoff) — often combined under a hierarchical supervisor, or orchestrator, that owns state, retries, and completion.
Sequential work passes an artifact forward. Parallel branches merge independent findings, while a routed handoff selects the relevant specialist.
Compare the chosen pattern with a single agent wherever both can perform the same task under comparable permissions. The additional roles should improve verified completion, reduce a specific failure, or enforce a required separation of authority.
| Decision point | Single agent | Multi-agent orchestration |
|---|---|---|
| Best when | One context, one permission set, and one verification pass can complete the task | Work splits along real permission, context, parallelism, or verification boundaries |
| Permissions and context | One agent holds every tool and the full history | Each agent gets only the access and context its role needs; the orchestrator passes a versioned handoff contract |
| Verification | The same agent checks its own work | An independent verifier can check production state before the coordinator closes the task |
| Cost and latency | Fewest model calls and no handoff overhead | More calls, handoffs, and retries — justified only if verified completion improves |
| Main failure mode | One agent makes an unchecked mistake | Context is lost, work is duplicated, or a failure cascades across handoffs |
| Choose it when | It passes the same verified-completion check as the multi-agent version | It beats the single-agent baseline on verified completion, a specific failure, or a required separation of authority |
If you are choosing the runtime that will execute these patterns, compare the leading agent orchestration tools and durable runtimes before you commit to a coordinator design.
Build better agents with Arize
Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.
Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.
Define a contract for each handoff
“Restore exports” gives the worker an objective but leaves the target and acceptance conditions unstated. Without the customer group, environment, and required result, the receiving agent has to supply assumptions that may change the task.
type RepairHandoff = {
schemaVersion: "v1";
incidentId: string;
target: "production";
affectedTenants: string[];
expectedPolicyVersion: number;
allowedChange: "restore export permission only";
completionEvidence: "production policy readback";
authoritativeReadbackRef: string;
evaluatorVersion: string;
};
type DecisionRecord = {
decision: "PASS" | "FAIL" | "REVIEW";
handoff_version: string;
evaluator_version: string;
evidence_refs: string[];
timestamp: string;
owner: string;
reason: string;
};These types define the handoff interface and persisted decision record, but your application must validate the payload before execution. A missing tenant list should send the worker back for clarification, while a proposed change outside the approved permission scope should be blocked by application controls.
Keep source artifacts or resolvable references with the contract so the implementation agent can inspect the policy diff and investigate evidence behind the instruction. Recording the contract version also helps distinguish a bad decision from a receiver interpreting an older schema.
A completion check in code
The following Python example compares an expected state with a readback fixture, recording a false completion when the worker claims success but the state disagrees. You can run it without external dependencies.
def evaluate_completion(expected, observed, claimed_success):
if observed is None:
return {"decision": "REVIEW", "reason": "missing_readback"}
mismatches = {
key: {"expected": value, "observed": observed.get(key)}
for key, value in expected.items()
if observed.get(key) != value
}
decision = "FAIL" if mismatches else "PASS"
return {
"decision": decision,
"reason": "state_mismatch" if mismatches else "verified_readback",
"false_completion": claimed_success and bool(mismatches),
"mismatches": mismatches,
}
expected = {
"tenant_group": "enterprise-eu",
"environment": "production",
"policy_version": 18,
"invoice_export_enabled": True,
}
observed = {**expected, "invoice_export_enabled": False}
result = evaluate_completion(expected, observed, claimed_success=True)
decision_record = {
"decision": result["decision"],
"handoff_version": "v1",
"evaluator_version": "policy-readback-v1",
"evidence_refs": ["policy-readback:run-284"],
"timestamp": "2026-10-19T00:00:00Z",
"owner": "incident-automation",
"reason": result["reason"],
}
assert decision_record["decision"] == "FAIL"
assert result["false_completion"] is True
print(decision_record)In a live workflow, populate observed from an authenticated read of the authoritative system after the action, and retain the timestamp and source record with the evaluator result. The fixture above demonstrates the comparison; production verification needs that actual readback.
Before the implementation agent acts, check that the handoff names the tenant group, approved scope, policy version, and source record. If any of those facts is missing, stale, malformed, or cannot be compared with live state, stop and send the case for review.
Save the contract version, evaluator version, source record, result, timestamp, and owner with every decision. PASS lets the approved change continue. FAIL creates a regression case. REVIEW sends the case to a person. These checks supplement security, approvals, rollback, monitoring, and legal review; they do not replace them.
Because the check proves only the fields it reads, your acceptance conditions may also require a user-facing export test or evidence that unrelated permissions remained unchanged. Add those checks before treating the result as sufficient to close the task.
Common multi-agent orchestration failures
Context gets lost between agents
Your investigator can identify the correct export policy while the next agent receives an incomplete summary. If the tenant group disappears during that transfer, the implementation agent may repair a different account.
The handoff carries the action forward but drops the tenant, environment, and permission limits.
In Arize AX, inspect the recorded inputs and outputs around the handoff. Capture the target and source reference explicitly in your instrumentation, then reject incomplete transfers before a worker can act.
Parallel agents duplicate work or make conflicting changes
Parallel work pays off when your assignments cover separate parts of the request. Vague boundaries can produce several competent answers to the same subtask while another part of the request remains untouched.
Anthropic’s research-system account describes this failure in practice. A broad semiconductor-shortage assignment sent one worker toward the 2021 automotive crisis, while two others duplicated research on 2025 supply chains.
If two agents read policy version 18 and each saves its own change, the later write can erase the first. Assign one writer or reject writes against an outdated version.
Agents keep handing off without making progress
A handoff loop consumes time while leaving the task unchanged. Your support agent may send an export incident to billing, which returns it to support because neither role has permission to inspect the policy.
A successful transfer event can hide the stall. In the illustrative trace below, both routes execute correctly, but the evidence version stays unchanged and nobody takes responsibility for the missing access.
00:00 support -> billing reason=export_failure evidence=v1
00:04 billing -> support reason=needs_policy evidence=v1
00:08 support -> billing reason=export_failure evidence=v1
00:12 coordinator pauses reason=no_progressInspect the routing sequence alongside each agent’s reason and evidence. Configure the coordinator to pause repeated routes when no useful information appears, then request the missing access or escalate to an authorized operator.
Microsoft’s handoff guidance identifies repeated bouncing between agents as a design risk. Choose a route limit for your workload and record why it was reached; legitimate investigation can require revisiting an earlier agent.
One agent’s failure spreads through the workflow
An upstream failure can become a downstream assumption. If your policy lookup times out and returns an empty payload, an implementation agent might interpret the missing policy as permission to create a replacement.
Preserve the difference between a successful lookup that found no record and a lookup that failed. Validate the tool result before releasing dependent work, then recover from the last confirmed step.
- Record the failed lookup with its error, target, and attempt identifier so the coordinator can distinguish unavailable evidence from an empty result.
- Pause the implementation agent because its required policy evidence is unavailable, while allowing unrelated read-only investigation to continue where appropriate.
- Retry the failed lookup within a bounded policy, or use an approved fallback that supplies comparable evidence for the same target.
- Resume dependent work only after validation succeeds, retaining the original failure and recovery attempt in the execution record.
The team declares success before completing the task
A research team can finish every assigned search and still deliver an incomplete comparison. If you requested four companies and the coordinator received evidence for three, fluent synthesis can hide the missing company.
Check completion against the original request before accepting the coordinator’s summary. This runnable Python fixture checks structured findings for missing company identifiers and source references, using only the standard library.
requested = {"company-a", "company-b", "company-c", "company-d"}
findings = {
"company-a": {"source_refs": ["source-1"]},
"company-b": {"source_refs": ["source-2"]},
"company-c": {"source_refs": []},
}
missing = requested - findings.keys()
unsourced = {name for name in requested & findings.keys()
if not findings[name].get("source_refs")}
if missing or unsourced:
print("INCOMPLETE", sorted(missing), sorted(unsourced))
else:
print("READY_FOR_CONTENT_REVIEW")The example prints INCOMPLETE, identifying company-d as missing and company-c as unsourced. Keep the task open and request those findings before synthesis; source presence alone still requires review for relevance and factual support.
Measure task success and coordination cost
- Verified completion rate: completed tasks that pass the independent outcome check divided by eligible tasks. Use it to compare whether a coordination change helps users.
- Critical coordination failures: counts of cross-tenant actions, unauthorized changes, duplicates, or false completion claims. Set separate release conditions for consequences that an average success rate must not hide.
- Handoff validity: accepted handoffs that satisfy required fields, schema versions, and authority checks divided by attempted handoffs. Use failures to improve the interface; a valid schema alone cannot establish that its evidence is correct.
- Cost and time per verified result: divide total execution spend, including failed attempts, by verified completions. Report median and p95 time to verified completion alongside the unresolved-task count. Use the result to decide whether the extra workers fit the workflow’s budget.
In an illustrative comparison, a candidate completes 18 of 20 test incidents against 16 for the baseline, but one candidate runs the wrong tenant. The higher completion rate therefore comes with a critical failure that the aggregate score hides. Preserve that case and test the fix against the same initial state before releasing the change.
Repeat the comparison when model behavior varies, keeping attempts grouped by task so a favorable run does not become the sole basis for release. Report pass@1 (single-attempt success) and pass^k (success on every attempt) separately: an agent team that changes external systems should be judged on pass^k and critical-failure rate, not on its best run. The agent reliability guide covers repeated-run consistency alongside outcome, recovery, and efficiency measures.
Choose the evidence each evaluator needs
- Handoff record: confirm that the receiving agent got the target, constraints, and source artifacts.
- Span: inspect one tool call’s arguments or a recorded approval.
- Trace: inspect the sequence of dispatches, tool calls, retries, and completion checks.
- Session: examine behavior that depends on earlier interactions or continues across several traces.
Use the evaluator scope documentation to select the appropriate unit, retaining task and attempt identifiers when work resumes asynchronously. Session grouping connects the activity, while the action record and readback establish what changed.
Turn failed runs into regression tests
If production still blocks exports, compare the intended tenant group with the handoff received by the implementation agent. A mismatch identifies a transfer to investigate and a contract change you can test through the workflow below.
A reviewed failure becomes a repeatable test. Compare both versions under the same conditions, then retain new production failures for the next run.
Use Arize AX tracing to inspect the model calls and tool activity around the failed transition, recording the handoff payload or artifact reference alongside the observed result. When earlier interactions supply necessary context, connect them through sessions.
Once you confirm the failure, create a dataset case from the relevant spans with the original request, initial-state fixture, and expected result. That fixture lets each version start from comparable conditions, preventing an already-repaired account from hiding the original bug.
Use Agent Experiments to compare the current coordinator with the proposed contract change on the failing case and related cases that previously worked. Check both whether the tenant survives the handoff and whether the final policy is correct, so fixing the transfer also improves the result.
After release, retain the case in regression coverage and monitor for recurrence so new failures can update the next comparison. Your orchestration service owns execution and permission enforcement, with tracing and evaluation supplying evidence for each proposed change. For a step-by-step workflow, see how to build agent evals from traces.
Frequently asked questions about multi-agent orchestration
When should you use multi-agent orchestration instead of one agent?
Use multiple agents when work can run independently, as when research workers investigate different markets, or when roles need separate permissions or context. Compare verified completion, cost, and latency with a single-agent baseline before adding coordination.
What should an orchestrator pass between agents?
Pass a versioned contract with enough information to act safely:
- Objective and expected output.
- Target, permissions, and required approval.
- Source artifacts, constraints, and state version.
- Completion evidence and escalation conditions.
Can an LLM judge evaluate a multi-agent workflow?
Yes, for semantic questions such as whether a synthesis follows its sources, provided the judge receives the report and supporting evidence. When the criterion concerns execution behavior, supply the relevant trace as well.
Use code or system reads for directly verifiable outcomes. Calibrate the judge against human-reviewed examples before its scores affect a release.
Should you evaluate outcomes or the full agent trajectory?
Evaluate both: the outcome check establishes whether the task succeeded, while the trajectory explains how it got there. A correct result can still involve an unauthorized action or an expensive retry loop, so assess completion alongside the constraints on execution.
What is the difference between a multi-agent system and multi-agent orchestration?
A multi-agent system is the full set of agents, models, tools, and memory. Multi-agent orchestration is the control layer inside it: task decomposition, assignment, handoffs, shared state, recovery, and the completion decision.
What are the main multi-agent orchestration patterns?
The main patterns are sequential (pass an artifact forward), parallel (independent branches that later merge), routed or handoff (send work to the relevant specialist), and hierarchical (a supervisor or orchestrator coordinates workers). Most production systems combine them.
How do you trace and debug a multi-agent workflow?
Trace the workflow as one end-to-end request: a root trace, an agent span for each worker, and parent-child links through every routing decision and handoff. Capture agent identity, handoff content, tool calls, shared-state changes, retries, and final ownership, then evaluate both the full trajectory and each handoff.
Start with one agent. Add another only when it has different access, can do independent work, or can verify a result the first agent cannot. Define what each handoff must carry, decide how you will prove the customer outcome, and keep real failures as regression tests. Then make the whole workflow observable: trace every handoff, evaluate outcomes and trajectories in production, and use those traces to decide the next coordinator change.
