The agent lifecycle is the full path from building an agent to running it in production and improving it over time. A practical lifecycle includes design, instrumentation, offline evaluation, staging tests, deployment gates, online evaluation, monitoring, error analysis, dataset curation, experiments, and iteration.
The important point is that deployment is not the finish line. Once an agent interacts with real users, real tools, and changing context, production behavior becomes the best source of truth for what to improve next. That is a different working model from shipping a deterministic service, where the tests you wrote before launch mostly tell you what will happen after it.
Do not confuse this with the agent control loop, which is the runtime cycle inside a single run, measured in seconds. The lifecycle is the operational cycle around the whole system, measured in weeks, and it includes the humans.
Build better agents with Arize
Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.
Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.
Key takeaways
- The lifecycle is a loop, not a line. Production traces feed error analysis, error analysis feeds the evaluation dataset, and that dataset is what gates the next release.
- Instrumentation belongs in the design stage. Adding tracing after an agent is already in production means every failure before that point is unrecoverable.
- The deployable artifact is larger than the model: prompts, tool schemas, retry policy, step limits, and retrieval configuration all change behavior and all need versioning.
- Deployment gates need a pass threshold on a curated set of cases, including the ones that already failed once, or every release re-litigates the same bugs.
- Retirement is a real stage. Credentials get revoked, tool access gets removed, and traces get archived, and nobody plans for it until an orphaned agent is still calling an API.
The stages
Design and scoping. Decide what the agent is allowed to do before deciding how it thinks. The tool list, the permission boundary, and the definition of a successful task constrain everything downstream. A task nobody can grade is a task you cannot improve.
Instrumentation. Emit a trace per run with a span per step, capturing the assembled context, the model call, tool arguments and results, and the terminal stop reason. This is the stage teams skip, and skipping it means every later stage runs on anecdote.
Offline evaluation. Run the agent against a fixed set of cases and score the outputs, plus the trajectory where the path matters. Early on this dataset is small and hand-written. That is fine. It improves by absorbing real failures.
Staging and pre-deployment testing. Exercise the tool surface against sandboxed or mocked systems, including the failure paths: a tool that times out, one that returns malformed data, a permission that is denied. Agent testing catches failures ordinary unit tests miss, because the interesting bugs live in the interaction between steps rather than in any single function.
Deployment gates. A defined rule for what score on what dataset permits a release. Without a gate, evaluation is a report nobody blocks on.
Online evaluation and monitoring. Score a sample of live traffic continuously and watch the operational signals: stop reason distribution, step counts, tool error rates, latency, and cost per run. A rise in runs terminating on the step limit is an early warning, usually arriving before quality complaints.
Error analysis. Read the failed traces and cluster them by cause rather than symptom. This is manual, and it produces more improvement per hour than anything else on the list.
Dataset curation and experiments. Turn each cluster of failures into cases, add them to the evaluation set, then change one thing and measure. Prompt, tool description, model, retrieval configuration, step limit. One at a time, or you will not know which one moved the number.
Retirement. Decommission deliberately. Revoke credentials, remove tool permissions, archive the traces you may need for audit, and confirm nothing upstream is still calling it.
What makes it different from the model lifecycle
Classic model lifecycle management assumes the artifact is a trained model, that you control retraining, and that offline metrics predict online behavior reasonably well. Agents break each of those assumptions.
The model is usually someone else’s and can change under you. The behavior you control lives in the scaffolding: prompts, tools, context assembly, and limits. Change is cheap, which sounds like an advantage and mainly means configuration drifts faster than anyone documents. Offline scores also generalize worse, because an agent’s failure often depends on a specific tool response your test set never produced.
The compensating advantage is speed. You can ship a fixed tool description in an hour. The constraint is verification, not implementation, which is why the lifecycle puts so much weight on evaluation and gating.
The loop that actually drives improvement
Strip the diagram down and one cycle does the work: production traces, error analysis, curated cases, an experiment, a gate, release, back to production traces.
Teams that run this loop deliberately improve faster than teams treating each bug report as a one-off prompt edit, because the failures accumulate into an asset instead of evaporating. The shift from ad hoc fixes to systematic agent improvement is mostly a matter of whether failures get written down in a form that can be re-run.
One caution: an evaluation set built entirely from past failures overfits to them. Keep a held-out slice of ordinary traffic in the dataset so a change that fixes the hard cases and quietly degrades the easy ones is visible before release.
Who owns which stage
The lifecycle crosses roles, and the handoffs are where it stalls. Engineers own instrumentation, the harness, and gates. Domain experts own what counts as correct, which means the rubric and a meaningful share of error analysis. Whoever runs the service owns cost and latency budgets. When nobody owns the definition of success, the evaluation stage produces numbers no one trusts enough to block a release on, and the loop degrades into shipping and hoping.
FAQ
What are the stages of the agent development lifecycle?
Design and scoping, instrumentation, offline evaluation, staging tests, deployment gates, online evaluation and monitoring, error analysis, dataset curation, experiments, and eventually retirement. Names vary between teams. What matters is that the post-deployment stages exist at all, since that is the half most versions of this process omit.
How is the agent lifecycle different from MLOps?
MLOps centers on training, versioning, and deploying a model you control, with retraining as the main improvement mechanism. The agent lifecycle centers on a system built around a model you often do not control, where improvement comes from changing prompts, tools, retrieval, and limits. Both need monitoring and gates. The agent version puts far more weight on tracing individual runs, because behavior varies per run rather than per model version.
When is an agent ready for production?
When you can answer three questions with evidence. What does success mean for this task, and can you measure it. What is the worst thing the agent can do, and what prevents it. When it fails at 2am, what will you look at. Passing an evaluation threshold matters, but only if the dataset behind it resembles real traffic.
How often should the improvement loop run?
Continuously for monitoring and online scoring, since those are automated. Weekly or per-sprint for error analysis and experiments, and faster right after a launch when the failure rate is still informative. The cadence matters less than whether each pass ends with new cases added to the evaluation set.
Does the lifecycle change for multi-agent systems?
The stages are the same, but attribution gets harder. A failed task has to be traced to the subagent and step that caused it, which raises the bar on instrumentation and makes session-level evaluation more important than per-response scoring.
MLOps
Machine learning operations (MLOps) is the set of practices for moving a model into production and keeping it working there. It covers the whole lifecycle, not a single tool or stage.
In practice, MLOps includes four kinds of work:
- Versioning: Track each model along with the data and code that produced it.
- Deployment: Package the model and release it to production.
- Monitoring: Watch how the model behaves after release.
- Retraining or rollback: Replace the model or return to an earlier version when its behavior drifts.
MLOps vs. model monitoring
Model monitoring is one MLOps activity. It means watching live inputs, outputs, and performance.
A team can monitor a model and still lack the release, versioning, and rollback practices that make the work MLOps. For more on monitoring, see model monitoring.
MLOps vs. LLMOps
LLMOps applies the same lifecycle practices to large language model (LLM) systems. In these systems, prompts, retrieval, and tool configurations change alongside model weights, or instead of them.
Both depend on visibility into production behavior. For more, see LLMOps and AI observability.
MLOps vs. evaluation
Evaluation is the measurement practice that gates each stage: before release, during rollout, and after changes. It tells you whether a version should ship.
MLOps is the machinery that ships the version, watches it, and replaces it. For more on evaluation, see AI evaluation and EvalOps.
How MLOps maps to the agent lifecycle
The agent lifecycle stages follow the same loop. You develop and evaluate, deploy with versioning and rollback, observe in production, and feed what you learn into the next version.
For more on that last step, see the agent feedback loop.
