Harness engineering Archives

A small watercolor painting of a dog-like figure standing on a horizontal line in front of a background filled with green binary digits (0s and 1s). The figure is loosely sketched in brown and tan tones, with an abstract, whimsical style on a white background.

Agent harnesses have an expiration date

A benchmark-driven look at why agent harnesses need adaptive finish logic as model behavior changes across Claude, GPT-4o, and Gemma.

10 minutes read

By RL Nabors | 10 minutes read

Harness engineering

Arize-branded slide with a dark purple gradient background and the title “Testing agents with traces, evals, and CI/CD.” On the right, a diagram illustrates navigating information-dense traces, showing a single trace with many spans and large data blocks (including an LLM iteration span and a data retrieval span). Arrows point to tools for querying and analyzing trace data, representing how engineers test and debug AI agents using traces and evaluation workflows.

AI agent evaluation: How to test, debug, and improve agents in production

Lessons from building and shipping Alyx, our AI agent

8 minutes read

By Sally-Ann DeLucia | 8 minutes read

Harness engineering

Swarm management of agent harnesses cover image

Swarm management in agent harnesses: owning long-running agents

As we have built our own harness management tools internally at Arize, and watched external systems like Devin @cognition start managing other Devins, managed agents at @AnthropicAI and long running

10 minutes read

By Aparna Dhinakaran | 10 minutes read

Harness engineering

A blog hero diagram titled “How agent harnesses manage limited context windows.” At the center is a bounded “Context Window” (~128K–200K tokens) divided into sections: System & Tools (fixed, always present), Conversation History (grows over time), File Contents (largest consumer, often capped per read with an example limit), Tool Results (accumulate quickly, often truncated and persisted), Subagent Responses (often isolated context, summarized back), and Session Memory (persistent summaries and decisions, searchable and retrievable). On the left, inputs flowing into the window include file reads (large reads can dominate context), tool calls (e.g., grep, bash, search), subagent outputs (forked task results), and session state (memory, summaries, checkpoints). On the right, a “Harness manages” panel lists controls such as capping file reads, truncating tool results, compacting history, evicting stale data, using search instead of loading everything, and restoring context after compaction. A reserved strip at the bottom marks the “Model output budget (reserved),” indicating space kept for generation. Below, a box labeled “LLM (reasoning engine)” notes that the model only sees what the harness allows through. A callout emphasizes that most tokens are not conversation and that the harness decides what stays.

Context management in agent harnesses: memory, files, and subagents

A version of this article originally appeared on X. Every agent harness runs into the same limit: the context window is too small for everything the model might want to remember….

13 minutes read

By Aparna Dhinakaran | 13 minutes read

Harness engineering

Arize AX

Learn

Insights

Company

Arize AX

Learn

Insights

Company

Blog

Agent harnesses have an expiration date

AI agent evaluation: How to test, debug, and improve agents in production

Swarm management in agent harnesses: owning long-running agents

Context management in agent harnesses: memory, files, and subagents

Subscribe to our resources and blogs

Arize AX

Learn

Insights

Company

Blog

Agent harnesses have an expiration date

AI agent evaluation: How to test, debug, and improve agents in production

Swarm management in agent harnesses: owning long-running agents

Context management in agent harnesses: memory, files, and subagents

Subscribe to our resources and blogs

Subscribe to The Evaluator