An agent workflow is the defined sequence of steps an agent follows to complete a task. A typical one includes assembling context, planning, retrieval, one or more tool calls, a check on the result, and final synthesis. The sequence can be fixed in code, chosen by the model at runtime, or a mix where some steps are mandatory and the rest are open.
The central trade is simple to state and easy to forget. Every step you specify is a step the agent no longer decides. That sounds like a loss of capability, and it is usually a gain in reliability. Most production systems that work are far more specified than their demos were, because the team found out which decisions the model should not be making.
This term gets used loosely. Some people mean any agent run, some mean a framework’s graph definition, some mean a business process that happens to involve an agent. Keep it concrete: a workflow is the path. Orchestration is the act of coordinating movement along that path. A multi-agent system is the set of participants. All three can be present, and none of them implies the others.
Key takeaways
- A workflow is the specified path an agent follows. The more of it you specify, the fewer runtime decisions the model makes, which is frequently the right call.
- Workflows sit on a spectrum from fully fixed code to fully model-chosen, and real systems mix the two step by step rather than picking a side.
- Fix the steps where a wrong order is expensive, where compliance requires an audit trail, or where latency and cost need a ceiling. Leave open the steps where the input genuinely varies.
- A workflow is visible in a trace as a span shape. Once you know the expected shape, deviations from it become something you can measure rather than notice by accident.
- Debugging an agent means finding which step in the workflow broke: bad plan, missing context, wrong tool, failed call, or weak final synthesis.
The specification spectrum
Fully fixed. Every step is code. Retrieve, then summarize, then format. The model fills in content but never chooses order. Testable like ordinary software. Fails when the input needs a step you did not write.
Fixed with agentic steps. The skeleton is code and one or two steps are open, such as letting the model pick which of four tools answers the question before a mandatory verification step runs. This is where a large share of working production systems live. It gives you predictable cost and latency with adaptability where it is actually needed.
Model-chosen with guardrails. The agent decides the sequence, but code enforces required steps, step limits, and permission checks. More capable, harder to test, and it requires that you can afford variable cost per request.
Fully open. The agent decides everything until it stops. Appropriate for research and exploratory tasks, rarely appropriate for anything with a service level objective.
There is no correct point on this line. There is a correct question: for this step, is the right choice knowable in advance. If yes, write it down. If no, let the model decide and instrument the decision.
What a workflow looks like in a trace
A workflow has a shape, and instrumentation makes that shape a measurable object. A run appears as a root span with child spans in sequence: an LLM span for planning, a retriever span, one or more tool spans, another LLM span for synthesis. Grouping related runs into a session lets you compare the shape across many requests. The vocabulary of spans, traces, and sessions is worth getting right early, because it is what makes questions like “how often does this workflow skip verification” answerable with a query rather than by reading logs.
Once the expected shape is known, several checks become straightforward:
- Step count distribution. A workflow that usually takes four steps and sometimes takes nineteen has a branch worth investigating.
- Missing mandatory step. The verification span is absent in 3% of runs. That is a correctness bug regardless of what the outputs looked like.
- Order violations. A write span before the approval span.
- Where time and tokens go. Usually one step dominates, and it is often not the one people assume.
Choosing what to specify
Fix a step when:
- Order has correctness consequences. Validate before write, check entitlement before read.
- Compliance needs an audit trail. “The model usually does this” is not a control.
- Cost or latency needs a ceiling. A fixed path has a predictable bill.
- The step is mechanical. Formatting, logging, schema validation. Nothing is gained by asking a model.
Leave a step open when the input genuinely varies in a way you cannot enumerate, when enumerating the branches would produce dozens of near-duplicate paths, or when the task is exploratory and the value is in finding an approach rather than executing a known one.
The honest version of this trade: teams usually start too open and specify more over time, driven by incidents. Reading the failures and asking which decision should never have been the model’s is the mechanism, and it is one of the failure classes that ordinary software tests do not surface because the code did exactly what it was written to do, which is why the AI agent handbook treats workflow shape as something you specify before you optimize prompts.
FAQ
What is the difference between an agent workflow and an agent?
An agent is the system where a model chooses actions. A workflow is the sequence of steps that system moves through. Every agent run traces out a workflow, whether or not anyone wrote it down. The interesting question is how much of it was decided in advance.
Is a workflow the same as orchestration?
No. The workflow is the path. Orchestration is the layer that decides what runs next and enforces retries, budgets, and termination. A workflow can be documented on a whiteboard. Orchestration is running code.
Can a workflow be fully deterministic if it contains an LLM call?
The control flow can be. The content cannot. You can guarantee that the same five steps run in the same order every time while the text produced at step three differs between runs. That is often the guarantee you actually need, and it is why fixing the path is such a common answer to reliability problems.
How do you version an agent workflow?
Treat the definition as code and record which version produced each run. When you change the path, the change should be visible in the trace, otherwise a comparison across two weeks is comparing two different systems. Prompt text, tool schemas, and step order all belong in that version.
How do you know when to add a step versus improve a prompt?
Look at where the failures cluster. If the agent has the right information and produces a weak answer, that is a prompt or model problem. If the agent skips something, does things out of order, or acts without checking, that is a missing step, and no amount of prompt editing makes it a guarantee.
