Self-improving agents are agents that use observed behavior and evaluation results to improve their prompts, policies, tool use, retrieval, or workflows over time. The useful version of this term is not an agent that magically trains itself. It is an engineered loop where the agent can detect failures, collect examples, test candidate changes, and compare results against a baseline.
In practice, self-improvement depends on observability, evaluation datasets, experiment tracking, and governance. Without those pieces, an agent is not improving. It is just changing.