Self-improving agents are agents that use observed behavior and evaluation results to change their own prompts, policies, tool use, retrieval, or workflows over time. The useful version of the term is not an agent that trains itself. It is an engineered loop where failures get detected, examples get collected, candidate changes get tested against a baseline, and nothing gets promoted without that comparison.
Here is the part most descriptions skip. Almost no system marketed as self-improving updates model weights. What updates is context: the system prompt, the examples the agent retrieves before it answers, the descriptions attached to its tools, the routing rules, and the notes it keeps about what worked last time. The model stays exactly as the provider shipped it. Everything around the model changes. That distinction determines what you can test, what you can roll back in one commit, and what an audit trail can show.
Key takeaways
- Self-improving means future behavior changes based on recorded outcomes from past behavior, on a timescale of days rather than within a single run.
- What changes is almost always context, prompts, retrieved examples, or tool policy. Weight updates are rare, slow to evaluate, and hard to reverse.
- Without a fixed baseline dataset and a before-and-after comparison, an agent that modifies itself is not improving. It is changing, and you cannot tell the difference.
- Reinforcement learning is not the normal implementation. Production reward signals are sparse, delayed, and noisy, and every episode costs money and touches a real user.
- Human review still sits at the promotion step in essentially every shipping system. Automation proposes; a person approves, or the change stays narrow and reversible.
What actually changes
Rank the surfaces by how fast you can change them and how easily you can undo it.
Prompt and instruction text. The cheapest surface and the one most loops touch. A failure category gets read, an instruction gets added or sharpened, the change is versioned like code.
Retrieved examples. Instead of editing instructions, add a corrected example to the store the agent pulls from at runtime. Behavior shifts because the input changed, not because the model did. This scales past prompt editing, since a prompt has a length budget and an example store does not.
Accumulated context and memory. The agent writes down what it learned about a task, a customer, or a codebase and reads it back later. Most of the durable value sits here, and so does most of the mess, because bad notes persist and compound. Structuring that history instead of logging it is the idea behind building an agent on a context graph, where disagreement between reviewers becomes part of what the agent reads next time.
Tool descriptions and selection policy. If the agent keeps picking the wrong tool, the fix is usually a clearer tool description or a hard routing rule, not a smarter model.
Model weights. Fine-tuning is real and sometimes correct, particularly for narrow formatting or classification behavior. It is also the slowest loop here: a labeled set, a training run, an evaluation pass, a deployment, and a previous artifact to redeploy if it goes wrong. Most teams get more improvement per week from the four surfaces above.
The loop that makes it improvement rather than change
Five steps, and skipping any of them breaks the claim.
- Detect. Something notices the failure: an eval score, a thumbs-down, an error span, a retry, an escalation. Detection runs off recorded traces, which is why tracing and evaluating the agent is a precondition rather than a nice-to-have.
- Collect. Group failures into categories and pull real examples of each into a dataset. A category with three examples is an anecdote.
- Propose. Generate a candidate change: new instruction, new example, new tool description, new routing rule.
- Compare. Run the candidate and the current version against the same fixed dataset and read both numbers. This separates improvement from motion, and it is the step most often skipped, because testing agents properly is slower than shipping the change and hoping.
- Promote. Ship under whatever gate the risk justifies, and keep the previous version one revert away.
An agent can do several of these. It can categorize its own failures, draft candidate prompts, and run the comparison. What it cannot credibly do alone is decide that the comparison was good enough.
Where the human still sits
Be precise here, because the marketing around the term is not. The common shipping shape is that automation detects and proposes and a person approves. In practice that often looks like an engineer reading a week of failed sessions on Thursday and editing a prompt on Friday, with the tooling doing the finding rather than the deciding. That is a real improvement loop. It is not an autonomous one.
Fully autonomous promotion does exist, but only under conditions that make it safe: a narrow change surface, a strong offline gate, a small blast radius, and a fast automatic rollback. Anyone describing an agent that broadly rewrites its own behavior in production without review is describing an aspiration, not a system you should copy. The reason is not caution for its own sake. An agent optimizing against its own eval will eventually satisfy the eval without getting better, which is evaluation drift arriving through the front door, and a human at the promotion step is currently the cheapest detector for it.
How it differs from the adjacent terms
Think in timescales. Self-healing happens inside a single run and measures in seconds: the agent hits an error and recovers without a developer. Self-improving happens across runs and measures in days, because past runs changed something. Closed-loop describes the topology, meaning outcomes reach the thing that changes behavior instead of stopping at a dashboard. An agent feedback loop is the plumbing carrying the signal.
FAQ
Do self-improving agents retrain themselves?
Usually not. The overwhelming majority accumulate better context, better prompts, or better retrieved examples while the underlying model stays fixed. Fine-tuning appears in narrow cases where the target behavior is stable and abundant labels exist, such as output formatting or domain classification. If a description of a self-improving agent does not say which surface changes, assume it is the prompt.
What is the difference between a self-improving agent and a self-healing agent?
Scope and timescale. Self-healing is recovery within one run: retry with corrected arguments, fall back to another tool, repair malformed output. The agent behaves the same way on the next request. Self-improving means the next request is handled differently because something in the system was updated based on what earlier runs showed.
What evidence should justify an automated change to an agent?
At minimum: a named failure category with a count, a dataset of real examples from it, a score for the current version, and a score for the candidate. Add a check on unrelated cases, since the most common outcome of a targeted prompt fix is a gain on the target category and a quiet regression somewhere else. Without the second number, you have a change with a story attached.
Can an agent improve itself without any human review?
Within a bounded scope, yes, for things like reordering retrieved examples or adjusting a routing threshold, where the change is reversible and the metric is unambiguous. Across the whole behavior surface, no shipping system does this responsibly today. The blocker is not model capability. It is that the agent’s evidence for its own improvement comes from evaluators that can be wrong in ways the agent has no way to notice.
