Harness engineering is the discipline of building and maintaining the agent harness: the code around the model that assembles context, defines and executes tools, enforces limits, handles retries and errors, manages state between steps, and decides when the run is over. It treats that scaffolding as the primary engineering surface of an agent rather than as glue around the interesting part.
The central claim is that most reliability work in an agent system happens in the harness, not in the model. You do not control the weights. You do control what goes into the context window, which tools exist and how they are described, what happens when a tool returns a 500, how many steps are allowed, and what the agent is permitted to do without a human present. Those are ordinary software decisions, and they determine most of what users experience.
A note on vocabulary. “Agent harness” is becoming common usage among people building agents. “Harness engineering” as the name of a discipline is newer and not yet standard, so expect to define it once when you use it outside your own team. The work it describes is what agent teams already spend most of their week doing.
Key takeaways
- The harness is everything around the model call: context assembly, tool definitions, execution, limits, retries, state, and stopping conditions.
- Most agent failures are harness failures. A tool description that does not say when *not* to use a tool causes more incidents than the model’s reasoning ability.
- A harness is not a framework. A framework is a library you adopt; a harness is the specific scaffolding you own and can change this afternoon.
- The agent harness and the evaluation harness share a word and are different systems. One runs your product, the other runs your evals.
- Harness changes are the fastest lever you have, because they ship like normal code and need no retraining.
What sits inside the harness
Context assembly. What the model sees on this step: system instructions, the relevant part of the conversation, retrieved documents, tool results from prior steps, and current state. Deciding what to include, what to summarize, and what to drop is the most consequential code in most agents, and it is where the majority of quality problems originate.
Tool definitions and execution. Names, descriptions, argument schemas, validation, timeouts, and what the tool returns on failure. The description is a prompt, so it is subject to the same care as any other prompt. A vague description gets the tool called in situations it was never meant for.
Control flow and limits. Step caps, token budgets, wall-clock timeouts, loop detection, and the rule that decides the task is complete. Without these, a confused agent burns budget until something else stops it.
Error handling and retries. Which errors are retried, with what backoff, and what the model is told about the failure. A raw stack trace produces different behavior than a sentence describing what went wrong and what to try instead.
State and permissions. What persists between steps and turns, and which credentials the agent holds when it acts. The architecture and controls that make up an agent harness are where those boundaries are actually enforced, because the model cannot enforce them.
Why the harness matters more than the model choice
Swapping models is the first thing teams try when an agent underperforms, and it usually produces a smaller change than expected. A strong model given a bad tool schema, a truncated context, and no stopping condition still fails. A weaker model with precise tool descriptions, well-scoped context, and sane retries often does the job.
The corollary makes model comparison tricky: the harness is part of what you are measuring. Change the scaffolding and the same model produces different results, so a model comparison has to hold the harness fixed and a harness comparison has to hold the model fixed. Otherwise you measured both at once and learned nothing about either.
The practical framing is that building a reliable agent is mostly a harness problem and that the harness is versioned, reviewed, and instrumented like the rest of your service. This puts it squarely inside the broader shift in what AI engineering involves: less model training, more systems work around models you did not build.
Harness versus framework versus evaluation harness
Framework. A library that gives you a default harness: an agent loop, tool calling conventions, memory abstractions. Useful for getting started, and the abstraction is the tradeoff. When the failure is in how context gets assembled, you have to reach into that layer, and a framework that hides it makes the fix harder.
Harness. The scaffolding as it exists in your codebase, including whatever the framework supplied plus everything you wrote around it. It is yours, specific to your product, and it changes when you change it.
Evaluation harness. The software that runs your evals: loads cases, invokes the system under test, applies scorers, records results. Same word, different system. The agent harness serves users; the evaluation harness measures the agent harness. Both get called “the harness,” and the fastest way to disambiguate is to ask which one is in the request path.
Failure modes this discipline exists to catch
- Tool descriptions that overlap. Two tools that could plausibly cover the same request get selected inconsistently, and accuracy degrades as the tool count grows.
- Context that grows without a policy. Every step appends, nothing is dropped, and by step 12 the instruction that mattered is buried under tool output.
- Silent truncation. Context exceeds the window and something upstream trims it, usually from the wrong end, and the agent behaves as if it never received the constraint.
- Retries with no ceiling. A transient tool error becomes a loop, and cost climbs while every span reports fine.
- No stopping condition. The agent has no definition of done, so it either quits early and claims success or works past the point of usefulness.
FAQ
Is harness engineering the same as prompt engineering?
Prompt engineering is a part of it. Harness engineering also covers tool schemas, context assembly, retries, limits, state, and permissions, most of which cannot be expressed in a prompt at all. If your only lever is the wording of the system prompt, you are working on one component of the harness and leaving the rest untouched.
How do I tell whether a failure is in the model or the harness?
Read the trace and check what the model was given before you question what it produced. If the retrieved passage was missing, the constraint was truncated, the tool returned an unhelpful error, or the schema let an invalid value through, the failure is in the harness. Model-level failures are the ones where the input was complete and correct and the output still was not.
Does this mean I should not upgrade my model?
Upgrade, and expect it to change the mix of failures rather than eliminate them. Stronger models pick tools more accurately and produce fewer malformed arguments. Dropped constraints, runaway loops, and unbounded context growth are structural and survive the upgrade, because they are properties of the code around the model.
