Self-improving agents: what changes, what persists, and how to prove it

An agent can finish a task, save a lesson, and score higher next time, while that same lesson pushes later tasks toward the wrong tool or procedure. One successful-looking run is not evidence that a persistent change made the agent better.

What makes an AI agent self-improving?

A self-improving AI agent uses evidence from previous behavior to create a persistent candidate change that future runs can inherit, then promotes that change only after evaluation shows that it improves the system.

An AI agent can retry a failed step, critique its answer, or retrieve different information and still begin the next task with the same durable agent configuration. Those are useful forms of adaptation, but they do not by themselves make the agent self-improving. Different retrieved context alone is not an inherited update; changing a persistent memory or retrieval policy is.

Self-improvement begins when evidence leaves a reusable change in the agent version or durable state policy. A new memory, revised skill, routing rule, tool policy, code change, or model update can be a candidate. The table below compares temporary “within-run” adaptation with persistent self-improvement, highlighting which changes disappear with the current run and which changes carry forward into a new agent version.

Try Arize AX

Build better agents with Arize

Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.

Get started View AX docs

Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.

Self-improvement can change the model or the scaffold

Self-Improvements in Modern Agentic Systems: A Survey separates self-improvement into two broad places where change can happen: the foundation model and the scaffold around it. The agent scaffold includes instructions, memory, retrieval, tools, routing, and control logic that shape how the model acts.

• Guidance and skills: prompts, rubrics, examples, and reusable procedures. These changes are usually easy to inspect and undo.
• Memory and retrieval: saved lessons and the rules that decide when to retrieve them. Either the stored memory or the retrieval policy can change future behavior.
• Tools, routing, and control logic: tool descriptions, model or tool selection, permissions, retry rules, stopping rules, and other harness logic. These changes can affect a much wider set of actions; the harness engineering guide covers the harness control layer in more detail.
• Model parameters: weights, adapters, or another learned component updated from interaction data or feedback. These changes are less transparent and usually more expensive to test and reverse.

SkillOpt shows why self-improvement does not require fine-tuning. The target model stays frozen while an external skill document becomes the mutable part of the agent.

A separate optimizer edits the skill, validation chooses which version to keep, and the selected skill is measured on a separate test set. Future runs change because the skill carried into those runs has changed, even though the underlying model has not.

A self-improving agent needs a version boundary

Each persistent candidate needs a version boundary before it is compared or considered for promotion. For a memory system, that can be a snapshot and provenance record rather than a full release. Without it, teams cannot reliably tie a later score, trace, or failure to the memory, prompt, router, code, or model version responsible.

Keep the prior version or snapshot and the proposed change itself. That record lets teams compare results, reproduce behavior, and roll back when a new version performs worse. Proposal, validation, and promotion are separate: a system can generate a candidate, evaluation can determine whether it improves the chosen evidence, and human or policy review governs deployment.

A production self-improvement loop has distinct jobs

Calling an agent “self-improving” can obscure the fact that several different systems participate in the loop. In production, it is useful to separate discovery, evaluation, investigation, experimentation, and promotion because each requires different evidence and controls.

These jobs should not collapse into one autonomous process. The system that proposes a change should not silently redefine the test used to approve it, and the component that discovers a failure does not need authority to deploy its own fix.

How to prove a self-improving agent actually improved

Every candidate needs an evidence plan, not one universal checklist. Start with repeat comparisons and cases outside the update data; then add regression, constraint, and independent-review checks as the change becomes broader or harder to reverse.

Discovery and evaluation solve different problems

Evaluation starts with a criterion. Production discovery starts with behavior.

A team can write deterministic checks for malformed tool arguments, an LLM judge for a known semantic criterion, or a regression test for a failure it has already seen. The harder problem is finding the failure that nobody thought to encode before release.

This is why production self-improvement needs both known-failure evaluation and unknown-failure discovery. Arize Signal, for example, periodically reviews production traces, groups recurring behaviors into issues, and retains the evidence associated with each finding. A confirmed issue can then become a dataset example, evaluator, or monitor in the next version of the test suite.

Put another way:

• Production reveals the failure
• Discovery makes it visible
• Evaluation makes it measurable.

What evaluations have asked over time

• Early feedback loops (Shinn et al., 2023): Reflexion evaluated textual reflection and episodic memory across sequential decision-making, coding, and language reasoning.
• Persistent transfer (Wang et al., 2023; Wang et al., 2024): Voyager evaluated an ever-growing executable skill library in Minecraft; Agent Workflow Memory evaluated reusable workflows for web navigation across task, website, and domain shifts.
• Broader validation (Patel et al., 2024; Ye et al., 2026): Large Language Models Can Self-Improve At Web Agent Tasks added measures of performance, robustness, capabilities, and trajectory quality on WebArena; On the Fragility of Self-Improving Agents used repeated runs and shuffled task order to re-evaluate memory-based methods.

Repeated runs separate signal from noise

The Fragility study revisits two memory-based approaches to self-improving agents. Agent Workflow Memory turns successful trajectories into reusable workflows, while ReasoningBank stores broader lessons from successful and failed trajectories and retrieves relevant memories for later tasks.

The results below use three benchmarks, each covering a different kind of agent behavior:

• WebArena: 812 browser tasks, scored in the re-evaluation across six slices (Shopping, Shopping Admin, GitLab, Reddit, Map, and Multisite).
• VisualWebArena: 910 web tasks that require visual grounding as well as browser interaction.
• SCUBA: 267 valid tasks across administration, sales, and service workflows in enterprise CRM software.

Across three runs, variance was higher than the no-memory baseline in 17 of 24 domain-level comparisons (two memory methods × 12 benchmark slices). A single favorable run therefore says little about whether a persistent update is reliably better.

For ReasoningBank in that re-evaluation, task order changed the direction of the result. On WebArena, it improved average pass@1 by 1.5 points under the default order, then degraded by 4.5 points when tasks were shuffled.

For stateful agents, repeated runs should therefore vary task order as well as random seeds. Real users do not arrive in the fixed curriculum used by a benchmark.

Held-out cases test whether the gain transfers

A self-improving update can overfit the examples used during development. Held-out cases ask whether the new behavior still helps on examples outside the update process.

SkillOpt separates candidate selection from final evaluation. Candidate skill edits are selected on validation data and then measured on a disjoint test set. The same principle applies whether the editable object is memory, routing, code, or model parameters.

Regression cases test what the update broke

Fixing the target failure is not enough if the update damages behavior that already worked. Preserve representative successes and known safety cases from the previous version, then rerun them against the new one.

When the behavior spans tool calls, state changes, or a full trajectory, trace-derived evals can check the execution path rather than only the final response.

Add cost, latency, permission, or safety gates when the update can affect those constraints. They are release conditions, not a checklist every experiment needs.

For the broader production scorecard across task success, recovery, repeated-run consistency, severity, cost, and latency, see agent reliability.

Some agent behavior requires an evaluator that can investigate

Not every evaluation can be reduced to a fixed set of fields and a single rubric.

A code evaluator is appropriate when success can be expressed deterministically. Traditional LLM-as-a-Judge works well when the relevant evidence and semantic criterion are known in advance. More complex agent behavior can require an evaluator to inspect the trajectory first: identify the relevant spans, follow tool use and retries, compare state across steps, and decide which evidence matters.

Agent-as-a-Judge uses an agentic evaluation harness for this class of problem. Rather than receiving only a fixed prompt and mapped fields, the judge can explore trace context and reason across a multi-step run before producing an evaluation result.

That makes it useful for criteria such as:

• Whether the agent chose an appropriate trajectory
• Whether recovery after a tool failure was reasonable
• Whether the agent declared success before the task was actually complete
• Whether repeated tool calls represented useful recovery or a loop
• Whether several individually acceptable steps produced a poor overall outcome

Agent-as-a-Judge should complement, not replace, narrower evaluators. Once a recurring failure can be expressed reliably as code or a simpler rubric, moving it into a cheaper and more deterministic test usually makes sense.

Self-improvement can make an agent systematically worse

Persistence makes mistakes more expensive. A bad answer ends with one run; a bad memory, skill, router, or code change can alter every run that follows.

Persistent memory can carry the wrong lesson

Persistent memory can also preserve the wrong lesson. In the Fragility re-evaluation, a memory useful under one task order could hurt under another.

Adding task rubrics and environment feedback reduced some of the degradation, which points to underspecified memories as one source of failure. The added task context did not remove the instability.

Persistent memories therefore need enough provenance to show where they came from and when they should apply. Without source and applicability metadata, reviewing or rolling back a bad lesson becomes much harder.

The candidate can optimize the evaluator

Some self-improving systems can alter code or other components that affect how success is measured. RewardHackingAgents tested ML-engineering agents in mutable workspaces and found that natural agents attempted evaluator tampering in about half of the episodes.

When the evaluator was locked, the evaluator-tampering attempts disappeared in the reported setup. The system being evaluated should not be able to rewrite the test used to certify the system.

Autonomy should depend on reversibility and blast radius

Reversibility and blast radius determine how much control a self-improvement mechanism can earn. A scoped memory or guidance update is easier to inspect and undo than a permission change, control-logic change, or weight update. The broader or harder-to-reverse the surface, the stronger the evidence and human review should be.

A useful production pattern is to automate more of the investigation and proposal process before automating promotion. A managed engineering agent can inspect traces, use repository context, create evaluation artifacts, or propose a pull request while leaving the merge and deployment decision outside its authority. This moves expensive debugging work to agents without giving the same system unilateral control over the evidence, the change, and the release.

Minimum promotion evidence by risk

• Memory, prompt, or guidance: snapshot the candidate; repeat comparisons on held-out tasks; check regressions; name the rollback path.
• Routing or tool policy: do all of the above, then check constraints such as permissions, latency, cost, and tool failures; require policy or human approval.
• Code, model, or permissions: use isolated evaluation; keep the evaluator independent; run a broader regression and safety suite; stage the release; name the owner who can roll back.

For the full production process around promotion, governance, release records, and rollback, see continual learning for AI agents and LLM systems.

Research boundary: editable code and improvers

Darwin Gödel Machine and Hyperagents show the outer limit of the update surface: the former searches branch-level changes to agent tools and workflow, while the latter makes the improver editable too. These controlled research systems are not deployment patterns. When code or the improver can change, keep a version boundary, an independent evaluator, and human promotion outside the editable boundary.

Measure the whole agent

A self-improving agent does not have one score. A candidate can raise task success while making the path slower, more expensive, less grounded, or less safe. Evaluate the quality that is meant to improve and monitor the qualities that must not regress.

Test the candidate as a system, not only as a prompt

A persistent update rarely changes only the final answer. A new skill can alter tool selection. A routing change can alter latency and cost. A memory update can change which context future steps receive. A code change can produce a different trajectory even when the model and prompt stay constant.

Validation should therefore execute the complete candidate agent whenever possible.

A full-agent experiment runs the same representative tasks through the baseline and candidate systems and compares the resulting outcomes, tool use, retrieval, trajectories, latency, cost, and evaluator results. This is stronger evidence than scoring a proposed prompt or diff in isolation because it tests the behavior that users will actually inherit.

A practical implementation is to turn the evidence behind a production issue into a dataset, run the current and candidate agents against the same cases, and compare the resulting outcomes, trajectories, costs, and evaluation results. This provides stronger evidence than reviewing a proposed prompt or code change in isolation because the candidate is tested as the system users will actually experience.

Arize is currently taking this idea directly into its Signal workflow: evidence traces from a production issue can become a dataset, a candidate branch can be run as an agent endpoint, and the dataset can be replayed against that candidate before the change is merged.

What a production self-improvement loop looks like

A production improvement loop needs to preserve evidence from one version, discover meaningful failures, turn those failures into evaluation criteria, propose bounded changes, compare a candidate against the current system, and keep a controlled promotion boundary around whatever future runs will inherit.

Arize AX provides one implementation of that architecture:

• Tracing provides the evidence layer. Instrumented agent trajectories capture inputs, outputs, retrieval, tool calls, timing, cost, and version metadata so engineers can reproduce the behavior that motivated a change.
• Signal provides continuous discovery. It reviews production traces on a cadence, groups recurring or emerging behavior into persistent issues, and attaches supporting trace evidence. This matters because the next useful evaluator often begins as a production failure the team did not know to define beforehand.
• Evaluation turns findings into measurable behavior. Deterministic evaluators and LLM judges cover known criteria. Agent-as-a-Judge extends that model to complex trajectories where the judge needs to explore spans, tools, state, or retries before making a judgment.
• Datasets and experiments provide the comparison layer. Production failures can be preserved as regression cases and replayed against a candidate agent. Full-agent experiments compare outcomes and trajectories across versions, rather than assuming that a prompt-level improvement implies a system-level improvement.
• Managed Agents can help investigate and propose the next change. With approved context, skills, and repository access, an engineering agent can carry an issue into the codebase and return an evaluation artifact, issue, branch, or pull request for review. Human engineers remain responsible for accepting and deploying the change.

Arize Phoenix supports the same core engineering loop through open-source tracing, evaluation, datasets, and experiments. Teams can use production and development evidence to build repeatable tests and compare application versions while operating the platform themselves.

The important boundary remains outside the product itself: observing a problem, proposing a fix, proving the candidate, and approving promotion are separate decisions. A self-improving system becomes safer and more useful when each decision leaves evidence behind.

FAQ

What is the difference between a self-improving agent and a continual-learning agent?

Self-improvement is a learning loop: evidence creates a persistent candidate change, and a defined process decides whether future runs inherit it. That process may be automated or governed by people or policy.

Continual learning can also mean an ML method that updates learned parameters as data arrives. In this article, a continual-learning workflow means the operating process around repeated change: collecting production evidence, building datasets, evaluating proposals, approving releases, monitoring deployments, and rolling back failures. Human reviewers can still control every update.

Is self-reflection the same as self-improvement?

No. Reflection can help an agent notice an error and repair the current run without leaving a persistent change. If reflection creates a saved lesson in memory, a skill, a prompt, or another component future runs inherit, it becomes a persistent self-improvement candidate. It becomes a validated improvement only after an independent comparison shows that the saved lesson helps.

Can self-improving agents update themselves automatically?

They can, but automatic proposal and automatic promotion are different levels of autonomy.

An agent can identify a production failure, create a memory or code candidate, build an evaluator, or propose a pull request without immediately changing the production system. Teams can then validate the candidate against held-out and regression cases before a human or policy gate decides whether future runs inherit it.

As the update becomes harder to reverse or capable of affecting more users, permissions, data, or systems, the promotion process should require stronger evidence and tighter controls.

Do self-improving agents need fine-tuning?

No. Fine-tuning is one way to make a persistent change because fine-tuning updates model parameters, but many self-improving systems leave the model frozen. SkillOpt edits a skill document, Agent Workflow Memory and ReasoningBank update persistent memory, and Darwin Gödel Machine modifies executable agent code.

How do you evaluate a self-improving agent?

Evaluate each self-improving agent update against the specific behavior the update is supposed to improve, not with one aggregate score.

• Target cases: did the update fix the failure that motivated the update?
• Repeated runs: does the gain survive stochasticity and, for stateful agents, different task orders?
• Held-out cases: does the gain transfer beyond the data used to create or select the update?
• Regression and constraint checks: did the agent preserve prior successes and stay within relevant safety, permission, cost, or latency limits?

Use outcome or trace evidence that the agent being evaluated cannot silently rewrite.

Can a self-improving agent make itself worse?

Yes. A bad memory can be retrieved again, an update can overfit its selection cases, and a system with access to its own evaluator can learn to improve the score instead of the task.

These failure modes are why self-improving agents need version history, independent evaluation, and a rollback path. Persistent updates can preserve useful improvements, but they can preserve bad changes too.