Context engineering is the practice of choosing what information a model sees on each call. In agent work, it decides whether the model has the instructions, tools, and evidence it needs to take the next step correctly.
For example, consider a support agent resolving a billing dispute. On step one, the agent sees a short instruction, the customer’s plan, and the disputed invoice. By step five, the full early conversation matters less than a short note saying “customer was charged after downgrade” and the policy passage that governs refunds.
Context includes everything in the model’s input: system instructions, examples, tool definitions, retrieved evidence, memory, and prior steps. The user’s message is only one part of it.
Build better agents with Arize
Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.
Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.
Why context engineering matters for agents
An agent rebuilds its input many times during one task. Each rebuild is a chance to include the right material or to miss it.
You can write a clear prompt and still get a wrong answer. The model may never have seen the relevant record. It may have seen it three steps too early and lost it in a long history. Or it may have been given twelve tools when it needed two.
Context engineering helps with:
- Keeping constraints in view: Make sure the rule that governs the current step is present at that step.
- Reducing distraction: Remove stale history and unused tools that add tokens without changing the decision.
- Improving tool choice: Expose only the tools the step needs, with descriptions precise enough to pick correctly.
- Debugging failures: Trace a wrong action back to what the model actually saw.
For the broader idea of what counts as model input, see context.
How context engineering works
The work is choosing pieces of context deliberately at each step. Early in a task, the input may hold only a short goal and a plan. Later it may need a tool result, a file excerpt, an error message, and a note about what has already been tried.
Most context is built from five recurring parts:
- Instructions: The durable rules, output expectations, and boundaries that should stay stable across steps.
- Task state: The current objective, completed sub-steps, open questions, and decisions already made.
- Evidence: Retrieved passages, tool outputs, or observations from the environment.
- Tools: The tool definitions exposed for this step, with clear descriptions.
- Examples: A small number of demonstrations, used when format or judgment is hard to describe in words.
The Phoenix guide to context engineering basics shows how these pieces fit together in a prompt. Anthropic’s guide to effective context engineering for agents makes the same distinction between assembling context and editing one instruction.
Both sources point to the same habit. Treat context as a designed input that changes over a task, not a fixed string.
Ordering and format
Where material appears matters as much as whether it appears. Put the instruction that governs the current step where the model will attend to it.
Keep evidence close to the question it supports. Use clear delimiters or structure so the model does not mistake a quoted document for an instruction.
Maintaining context over time
Context grows as a task runs. When it does, summarize or drop stale material rather than appending forever.
Keep a record of what you dropped. A later step may need to retrieve it. For more on shrinking long histories, see context compaction.
Worked example: a billing dispute agent
Here is how the support agent’s context changes across a task.
On step one, the context holds a short system instruction, the customer’s current plan, and the disputed invoice. That is enough to decide the first action, which is to look up the charge.
On step three, a tool has returned a proration record. The invoice excerpt can shrink to the relevant line. The tool result should appear verbatim. The agent’s working note should record “customer was charged after downgrade, refund rule to check.”
The following block shows the shape of the step-three context. It is illustrative, not verbatim product output.
Step 3 context (illustrative shape, not verbatim product output)
- Instruction: resolve the dispute; cite the policy passage you rely on.
- Task state: downgrade confirmed 12 May; charge posted 14 May; refund rule unchecked.
- Evidence: proration record lines 4-6; invoice line 2.
- Tools available now: get_policy, issue_refund, reply.
- Do not include: full chat transcript from steps 1-2 (summarized in task state).
On step five, the agent needs the working note and the refund policy passage. The full early conversation is less useful than either.
The specific fields are not the point. Each step’s input was chosen for that step’s decision, and a reviewer can see why.
How to debug context
Start from the trace, not the template. A trace shows the rendered input for each call after variables, retrieval, and history handling have run. It also shows the output and any tool calls.
Reading traces as the record of what the agent actually did lets you ask concrete questions:
- Instruction survival: Did the governing instruction survive to the step where it mattered?
- Evidence placement: Was the evidence present, current, and near the question?
- Tool descriptions: Did a tool description invite the wrong choice?
A practical sequence works like this. Reproduce the failing step and read its exact input. Form one hypothesis, such as “the refund policy was absent” or “three similar tools competed.” Change only that element and rerun the same tasks.
If you change instructions, evidence, and tools at once, you cannot say which change helped.
For more on watching this in production, see AI observability.
How to measure whether a context change helped
Measure task outcomes on a fixed set of cases. Token count alone does not tell you whether the change worked.
Useful measures include:
- Task completion: Did the agent finish the task?
- Step correctness: Did the step you changed now produce the right result?
- Tool-selection accuracy: Did the agent pick the right tool, where that was the failure?
- Cost and latency: Did these stay within acceptable limits?
Include cases where extra context hurt in the past. Segment results by step type so an average gain does not hide a specific loss. For more on judging results, see LLM evaluation.
Limitations
Context engineering has real trade-offs, and some problems sit outside it.
- Missing capability: It cannot fix a model that lacks a skill, a tool that does not exist, or evidence that is unavailable.
- Cost and dilution: Longer context costs more and can dilute the model’s attention.
- Over-trimming: Aggressive trimming can remove the one constraint a later step needed.
- Summary drift: A summarized history can quietly turn a tentative observation into a stated fact.
- Bad evidence: Retrieved evidence can be stale or contradictory. Putting it in context does not make it true.
- Permissions: Actions that need approval still need checks outside the prompt.
Context engineering vs. prompt engineering
Prompt engineering usually means improving the wording of an instruction or example for a call. Context engineering is broader.
Context engineering decides whether that instruction should be present at all. It also decides which tools and evidence go with it, how prior steps are represented, and when old material should be removed.
Context management is a related term, but it is not always a synonym. Some teams use it narrowly for managing history length or memory stores. Context engineering covers that maintenance plus the initial selection, ordering, and formatting.
If your team uses the terms differently, the useful question is the same. Who decides what the model sees at each step, and how is that decision tested?
Frequently asked questions
Is context engineering just a new name for prompt engineering?
No. Prompt engineering improves the wording of instructions and examples. Context engineering decides the full input for each call and how that input changes as a task progresses. Prompt wording is one part of that larger decision.
How much context should I include?
Include material that could change the next decision, in a form the model can use, and leave out the rest. If removing a block never changes behavior on your test tasks, it is probably not earning its tokens.
If a recurring failure traces to a missing fact or constraint, add that material at the step where it is needed.
When should I summarize or drop history?
Summarize when the verbatim history is long, stale, or distracting, and you can keep the critical state in a shorter record. That record should hold decisions made, facts confirmed, actions taken, and open questions.
Keep the original retrievable where possible. Test that the shorter form does not lose constraints that matter later.
How do I know a context change worked?
Run the same fixed tasks before and after the change. Compare task outcomes and the specific step you targeted, and check that cost and latency did not get worse.
A change that only makes the template look cleaner has not been shown to work.
Putting context engineering to work
For each step in your agent, write down what the model needs to decide and what context supports that decision. Then compare that list with what the trace shows the model actually received.
Keep a fixed set of tasks for testing context changes, and change one element at a time. Tools such as Arize AX and Arize Phoenix let you review the rendered input for each call and attach evaluations to outputs. The Arize AX documentation covers how to set this up.
