Context compaction replaces a long agent history with a shorter record of the work so far. It lets an agent keep working within its context limit without losing the state the task still depends on.
For example, suppose a research agent has spent fourteen steps comparing two vendors’ data-retention terms. Its history holds three abandoned search strategies, two dead-end pages, and four useful sources. After compaction, step fifteen starts from a short record of the goal, the citation rule, the confirmed facts, and the one contract clause still to find.
Build better agents with Arize
Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.
Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.
Why context compaction matters for agents
Every agent step adds instructions, tool results, and model outputs to the history. The next model call usually resends much of that history, so it keeps growing.
Without compaction, a long task eventually fills the context window. Before that point, each step costs more, and the model has more stale or superseded material to weigh.
Context compaction helps with:
- Running long tasks: Continue past the point where the full transcript would no longer fit.
- Controlling cost: Send a short record on each step instead of the entire history.
- Reducing distraction: Remove abandoned approaches and dead ends that the model no longer needs to consider.
- Closing out phases: Record the result of a finished sub-task cleanly before starting the next one.
For long-running agents, compaction is often what makes a long task possible at all. Without it, the task ends when the window fills, or it degrades as the window fills with its own leftovers.
How context compaction works
At some point in a task, the system pauses and generates a condensed record of the work so far. The agent then continues from that record instead of the full transcript.
A good compacted record is more structured than a paragraph summary. It usually keeps these elements as separate fields:
- Goal and constraints: What the user asked for and any rules they set.
- Decisions: What was decided, with a brief reason.
- Confirmed facts: Results verified by tools or sources, with identifiers such as file paths, record IDs, URLs, or error strings kept verbatim.
- Actions taken: What the agent already did and what happened.
- Current state: Where the work stands right now.
- Open questions: What remains to be done next.
Material that is cheap to fetch again can be replaced by a pointer. A large file the agent can reread, for example, does not need to stay in the record.
Anthropic’s context engineering guidance and the Phoenix context engineering basics both treat history management as one part of deciding what the model sees at each step.
For more on what goes into a model’s input, see context and context engineering.
When to trigger compaction
Common triggers are a token threshold, a step count, or a natural break such as a sub-task completing.
Triggering at a natural break is usually safer. The record can close out a finished phase cleanly. Compacting in the middle of an investigation is riskier, because it is hardest to judge which half-finished threads will matter.
Compact before the window is full. If you wait until the limit forces the system to drop text, you lose control over what is kept.
Worked example: compacting a research task
Return to the research agent from the opening. After fourteen steps, a well-compacted record might look like this:
Goal: Compare vendors A and B on data-retention terms for the security review.
Constraints: Cite contract or policy text for every retention claim.
Confirmed: A retains deleted data 30 days (policy page, "Data retention" section).
Confirmed: B's public page states no period; contract text not yet found.
Done: Reviewed A policy and B marketing pages (full text re-readable from sources).
Open: Find B's retention period in contract text; then draft comparison.
Discarded from context: first two search strategies and pages ruled out as marketing summaries.
The agent continues with its task-critical state intact, using a fraction of the tokens.
Now consider a badly compacted record: “Researched vendor retention policies, found some information.” From that, the agent would likely reread A’s policy. It might drop the citation rule. It might also state a retention period for B that it never confirmed.
The difference between the two outcomes comes entirely from what the record keeps.
How to test context compaction
Treat compaction like any other change to your system. Run the same tasks before and after, and score them the same way as your uncompacted baselines.
Use tasks long enough to trigger compaction more than once. Then look for three specific failures:
- Lost constraints: A rule stated early in the task is broken late in the task.
- Repeated or contradicted facts: The agent investigates a confirmed fact again, or states the opposite.
- Altered identifiers: Names, IDs, or version numbers change slightly in the rewrite.
These failures rarely show up in average quality scores. Add explicit checks for them.
Check that required items survive compaction
The following script lists items that must appear in every compacted record. It checks the good record from the worked example and the vague one-line summary.
record = """Goal: Compare vendors A and B on data-retention terms for the security review.
Constraints: Cite contract or policy text for every retention claim.
Confirmed: A retains deleted data 30 days (policy page, "Data retention" section).
Confirmed: B's public page states no period; contract text not yet found.
Open: Find B's retention period in contract text; then draft comparison."""
must_keep = [
"Cite contract or policy text",
"30 days",
"B's public page states no period",
"Find B's retention period",
]
for item in must_keep:
status = "kept" if item in record else "MISSING"
print(f"{status:8} {item}")
vague = "Researched vendor retention policies, found some information."
lost = [item for item in must_keep if item not in vague]
print(f"vague summary loses {len(lost)} of {len(must_keep)} required items")
Output:
kept Cite contract or policy text
kept 30 days
kept B's public page states no period
kept Find B's retention period
vague summary loses 4 of 4 required items
A string check like this only confirms that an item is present. You still need to check that the agent honors the constraint in the steps after compaction.
Inspect the record in traces
Inspect the compacted record directly, as you would a prompt. Your traces should show the history before compaction, the record produced, and the steps taken afterward.
When a step after compaction goes wrong, start with a simple question. Was the needed fact present in the record, present but buried, or lost?
That kind of visibility is a basic AI observability requirement for any system that rewrites its own inputs.
Limitations of context compaction
Compaction is lossy, and the losses are chosen under uncertainty. At compaction time, the system cannot know for certain which detail a later step will need.
- Uncertainty gets erased: A tentative tool result such as “appears to be X” can become “is X” after rewriting. Every later step then treats that error as fact.
- Losses compound: Repeated compactions summarize earlier summaries, so each round can lose a little more.
- It has a cost: Generating the record takes a model call.
- Timing can be wrong: A trigger can fire at a bad moment, in the middle of a half-finished investigation.
The standard mitigations are to keep the original history retrievable, carry identifiers verbatim, and keep uncertainty labels such as “appears” or “not yet confirmed” through the rewrite. None of these removes the trade-off entirely.
Context compaction vs. truncation, memory, and summarization
Compaction is often confused with three nearby ideas.
Truncation drops the oldest material by position and hopes nothing important was there. Compaction replaces history with a deliberately composed record.
Long-term memory stores selected information across sessions for later retrieval. Compaction manages the working context inside one ongoing task. A compacted record may still be saved later as part of a task’s history. For more on persisting state, see agent state management.
A summary for a human reader aims to be easy to read. A compacted record aims to support the agent’s next decision. That is why verbatim identifiers and explicit open questions matter more than smooth narrative.
Frequently asked questions
When should an agent compact its context?
An agent should compact at a planned threshold, before the window is full. Good thresholds are a token budget, a step count, or a natural break between sub-tasks. Waiting until the limit forces truncation removes your control over what is kept.
What should a compacted record always keep?
A compacted record should keep the goal, user constraints, decisions with brief reasons, confirmed facts with identifiers and sources, actions taken with outcomes, current state, and open questions. Copy identifiers exactly rather than paraphrasing them.
Is context compaction the same as summarizing the conversation?
Not quite. A summary is written to be read, while a compacted record is written to be worked from. It keeps verbatim identifiers, explicit uncertainty, and open next steps in a structured form for the agent.
Can context compaction lose information the task needed?
Yes, and that risk is central to the technique. Reduce it by keeping task-critical fields explicitly, keeping the original history retrievable, and testing long tasks for lost constraints and contradicted facts after each compaction.
Putting context compaction to work
Define the fields your compacted record must contain before you write the compaction prompt. Goal, constraints, confirmed facts, and open questions are a good starting set.
Keep the full history stored somewhere the agent or a reviewer can reach, so any claim in the record can be checked.
Build a test set of tasks long enough to compact at least twice, and check required items after each compaction. Tracing tools such as Arize Phoenix can show the history before compaction, the record produced, and the steps that followed, so you can find where a fact was lost.
