Agentic memory is long-term memory for an LLM agent that organizes itself instead of sitting in a flat buffer. A-MEM is the specific research approach that gave the idea its name: rather than appending raw conversation turns to a growing blob and hoping semantic search finds the right slice later, the agent writes structured memory notes, links each new note to related existing notes, and revises older notes as new information arrives. The paper that introduced it is titled “A-MEM: Agentic Memory for LLM Agents.”
The practical difference shows up when an agent needs something from three sessions ago. A conversation buffer either still holds those turns or it does not. A structured store can be traversed: find the note about this customer’s billing setup, follow its links to the note about the failed migration, pull both into context. Whether traversal beats plain vector search depends heavily on the task, and the field has not settled that.
Build better agents with Arize
Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.
Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.
Key takeaways
- Agentic memory means an agent maintains persistent, structured, self-organizing memory rather than relying only on what fits in the current context window.
- A-MEM stores memory notes carrying context, keywords, and tags, links them to related notes, and updates existing notes when new ones arrive. That update step is often called memory evolution.
- Three kinds of memory get conflated: short-term working context, episodic memory of past runs, and semantic memory of learned facts. They have different retention rules and different failure modes.
- Memory systems accumulate stale and contradictory entries. Eviction and conflict resolution are the unsolved parts, not storage.
- Memory failures show up in traces before they show up in outputs, so instrumenting reads and writes is the prerequisite for debugging them.
Why flat memory runs out
The default memory design for an LLM application is the transcript. Keep the turns, truncate when the context window fills, and optionally summarize the middle. The ways it breaks are predictable.
Truncation drops the fact you needed. Summarization compresses away the identifier, timestamp, or exception message that mattered. Retrieval over transcript chunks returns three near-duplicates of the same exchange and none of the correction that came after. A bigger context window changes the economics without fixing the retrieval problem.
The deeper issue is that a transcript is a log, and a log has no notion of what is still true. It records that a user said their region was us-east-1 in March and nothing about their migration in June.
How A-MEM structures memory
The A-MEM design borrows from the Zettelkasten note-taking method, where atomic notes gain value through the links between them rather than through their position in a hierarchy. Applied to an agent, the pattern looks like this.
Write a note, not a turn. When something worth remembering happens, the agent generates a note containing the content plus descriptors: a short context summary, keywords, and tags. The note is a unit of knowledge, not a unit of dialogue.
Link it to neighbors. The new note is connected to existing notes it relates to, in both directions, so recall can start anywhere in the graph and walk outward.
Let old notes change. Adding a note can trigger revision of the notes it links to, updating their descriptors or their content. This distinguishes agentic memory from a write-once vector store, and it is where the design gets risky: an automated rewrite of memory is an automated chance to corrupt it.
Retrieve by traversal, not just by nearest neighbor. Recall can combine similarity search with graph structure, which is how a query about a deployment can surface an incident note that shares no vocabulary with it.
Short-term, episodic, and semantic memory
Most confusion about agent memory comes from collapsing three separate things into one word.
Short-term or working memory is what is in the prompt right now: the current task, recent tool results, the scratchpad. It is bounded by the context window and it should be aggressively pruned.
Episodic memory is the record of past runs: this task, these steps, this outcome. It is the substrate for learning from experience, and what agent workflow memory (AWM) mines to extract reusable procedures. The two are complementary: A-MEM organizes what the agent knows, AWM captures how it gets things done.
Semantic memory is distilled fact: this user prefers metric units, this API returns cursor-paginated results. It should outlive any particular session.
Deciding what gets promoted between layers is the hard design question. None of these layers is execution state, which is tracked separately.
Where memory systems break
Staleness. Nothing in a memory store expires on its own. An agent that learned a policy in Q1 keeps applying it in Q4 unless something overwrites the note. Few teams have a real eviction strategy, and “keep everything and let retrieval sort it out” is not one.
Contradiction. Two notes can both be well-formed and mutually exclusive. When both land in context, the model picks one, usually the one that appears later or reads more confidently. Most implementations have no principled arbitration.
Poisoning. A wrong fact written to memory is durable in a way a wrong answer is not. It will be retrieved again, and it will look authoritative because it came from memory. Automated memory evolution makes it worse by propagating the error into linked notes.
Retrieval that is confidently irrelevant. Memory recall is retrieval, so it inherits every retrieval failure mode. High similarity is not relevance. A memory can score well and still be the wrong memory.
Cost and latency. Note generation, link construction, and note revision are extra model calls on the write path. They are easy to underestimate because they happen when no user is waiting, right until they land on the critical path.
Every one of these is diagnosable from traces, provided memory reads and writes are instrumented as spans alongside the model and tool calls. That is why memory work and agent evaluation belong together.
FAQ
How is A-MEM different from a vector store?
A vector store is flat and write-once: you embed a chunk, search for nearest neighbors, and the stored items never change. A-MEM adds descriptors, explicit links between notes, and revision of notes as new ones arrive. A vector index is usually still underneath, doing the similarity lookups.
Do agents need long-term memory if context windows keep growing?
A larger window helps and does not remove the need. Filling it with everything the agent has ever seen raises cost and latency on every request, and models still lose precision on facts buried in the middle of long inputs. Memory is about deciding what belongs in the prompt, not how much fits.
What is the difference between agentic memory and agent workflow memory?
Agentic memory organizes knowledge: facts, entities, and their relationships. Agent workflow memory captures procedure: sequences of actions that worked before and can be reused. Background on both sits in the AI agent handbook.
Should I build this or use a memory library?
Start by writing the memory reads and writes yourself, even badly, because the design questions that matter are yours: what is worth remembering, what expires, and who wins when two notes disagree. A library gives you the storage and the retrieval, which is the part that was never hard. It will not decide your eviction policy. Whichever way you go, instrument the reads and writes as spans first, because you cannot evaluate a memory system you cannot see.
How do I tell whether memory is helping or hurting?
Compare runs with memory enabled against runs without it on the same tasks, and evaluate the trajectory rather than only the final answer. Track whether retrieved memories were used, whether any was contradicted by the current session, and what memory added to token spend. Approaches are in AI agent testing.
Episodic memory
Episodic memory is an agent’s record of specific past runs: what happened, in what order, and how it turned out. It lets an agent reuse what it tried before when a similar situation comes up again.
An episode usually contains the task or request, the steps taken, the tools called, the context at the time, and the outcome. The outcome includes whether the run worked.
Episodic memory vs. semantic memory
Semantic memory keeps a general fact, such as “this customer’s plan is EU.” Episodic memory keeps the event that the fact came from. For example: “On 12 March the agent attempted the plan change, the billing tool rejected it, and support completed it manually.”
This difference matters when you retrieve. Pull an episode when a similar situation recurs and the sequence of what was tried is useful. Pull a semantic fact when you only need the fact.
Keeping episodes inspectable
In the A-MEM research system described in this entry, episodes are the retained runs that later memory operations organize and link.
In a production agent, link each episode to the trace it came from. A reviewer can then see the actual steps instead of a summary. Record provenance when you write the episode: when it happened, which user or session it came from, and what outcome check was applied.
For more on how this state is stored and checkpointed between runs, see agent state management.
Worked example
Suppose a support agent fails to process a refund. The order lookup tool needs an order ID that the customer has not given.
The stored episode is not “refunds sometimes fail.” It is the whole run: the request, the failed lookup, the missing input, the clarifying question, and the successful retry. A later run with the same gap can then recover the same way instead of failing again.
To check that retrieval helps, build test cases where a past episode should be reused and cases where it should not. Run them with AI evaluation. Then inspect the trace to confirm the agent used the episode it retrieved.
Failure modes to test
- Stale episodes: A procedure that worked before a tool or policy change may now be wrong. Episodes need an expiry or a way to be replaced, not endless accumulation.
- Wrong-episode retrieval: An episode that looks similar on the surface can push the agent into the wrong action. Use the outcome and context stored with the episode to decide on reuse, not text similarity alone.
Agent Workflow Memory (AWM) is a related pattern. An episode is the raw retained run. A workflow memory is a proven procedure generalized from many episodes.
Semantic memory
Semantic memory is an agent’s store of general facts and preferences that hold beyond any single run. It lets the agent use what it already knows without asking again.
Examples include a customer’s plan and region, a user’s preferred format, or a product rule confirmed across several interactions.
How semantic memory relates to episodes
Episodic memory keeps the specific run. Semantic memory keeps the fact distilled from it. Keep the two linked.
Each fact should point back to the episode or source it came from, along with when it was learned. A reviewer can then check the fact instead of trusting it.
In the A-MEM research system described in this entry, semantic facts are part of how retained experience is organized for later retrieval.
Deciding what to store
In a production agent, the write decision is where you control quality. Not every statement in a conversation should become a durable fact.
Promote a statement to semantic memory when the user states it as stable, confirms it, or repeats it. For example, “our billing entity is in the EU” is a stated, stable fact.
Keep inferred guesses out, or mark them as unverified. A wrong fact retrieved with confidence is harder to catch than a missing one. For more on persisting, versioning, and expiring this state, see agent state management.
Worked example
Suppose that across three support sessions a customer says their team works in Pacific Time and wants summaries by email. Stored as semantic memory, that becomes two facts:
- Timezone: Pacific, with the source session noted.
- Summary channel: email, with the source session noted.
The next run uses both facts without asking again. The episodic record of each session still keeps the full run.
Now suppose the customer later corrects the timezone. You update the semantic fact. The older episodes stay as history, not as a competing current answer.
You can check in the trace whether a retrieved fact helped or a stale one misled the agent. You can score that behavior with AI evaluation.
Proven procedures generalized from many facts and episodes belong to a different pattern, Agent Workflow Memory (AWM). They are not semantic memory.
Failure modes to test
- Staleness: Plans, regions, and preferences change. Semantic entries need update and expiry paths, and a correction must stop the old value from being retrieved.
- Over-generalization: A one-time request such as “expedite this once” can be stored as a standing rule such as “always expedite.” The agent then does something the user never approved.
- Conflict: Two stored facts can disagree. Set a clear rule, such as “most recent wins” or “higher-authority source wins,” and stop retrieving the losing value.
Test all three as behavior cases, not storage checks. The question is what the agent was given and what it did, not what the store contains.
