Arize:Observe is June 4
Register now

  1. Arize AX
  2. Phoenix OSS
  3. Pricing
  4. Docs
  5. Learn
  6. Company
Book a demo Sign in Sign up

Blog

Filter
close Clear all

What we learned testing 7 models under the same agent harness

Model swaps look like configuration changes, but they behave more like product migrations. A new model may be cheaper, faster, easier to get capacity for, or stronger on public benchmarks….

9 minutes read
By Nancy Chauhan | 9 minutes read
Harness engineering

Coding agent tracing and evaluation: An open source tool to improve AI coding workflows

Announcing coding harness tracing for observing, evaluating, and improving coding agent workflows across Claude Code, Cursor, Codex, GitHub Copilot, and Gemini CLI.

4 minutes read
By Duncan McKinnon | 4 minutes read
Harness engineering
A small watercolor painting of a dog-like figure standing on a horizontal line in front of a background filled with green binary digits (0s and 1s). The figure is loosely sketched in brown and tan tones, with an abstract, whimsical style on a white background.

Agent harnesses have an expiration date

A benchmark-driven look at why agent harnesses need adaptive finish logic as model behavior changes across Claude, GPT-4o, and Gemma.

10 minutes read
By RL Nabors | 10 minutes read
Harness engineering
Arize-branded slide with a dark purple gradient background and the title “Testing agents with traces, evals, and CI/CD.” On the right, a diagram illustrates navigating information-dense traces, showing a single trace with many spans and large data blocks (including an LLM iteration span and a data retrieval span). Arrows point to tools for querying and analyzing trace data, representing how engineers test and debug AI agents using traces and evaluation workflows.

AI agent evaluation: How to test, debug, and improve agents in production

Lessons from building and shipping Alyx, our AI agent

8 minutes read
By Sally-Ann DeLucia | 8 minutes read
Harness engineering
Swarm management of agent harnesses cover image

Swarm management in agent harnesses: owning long-running agents

As we have built our own harness management tools internally at Arize, and watched external systems like Devin @cognition start managing other Devins, managed agents at @AnthropicAI and long running

10 minutes read
By Aparna Dhinakaran | 10 minutes read
Harness engineering
A blog hero diagram titled “How agent harnesses manage limited context windows.” At the center is a bounded “Context Window” (~128K–200K tokens) divided into sections: System & Tools (fixed, always present), Conversation History (grows over time), File Contents (largest consumer, often capped per read with an example limit), Tool Results (accumulate quickly, often truncated and persisted), Subagent Responses (often isolated context, summarized back), and Session Memory (persistent summaries and decisions, searchable and retrievable). On the left, inputs flowing into the window include file reads (large reads can dominate context), tool calls (e.g., grep, bash, search), subagent outputs (forked task results), and session state (memory, summaries, checkpoints). On the right, a “Harness manages” panel lists controls such as capping file reads, truncating tool results, compacting history, evicting stale data, using search instead of loading everything, and restoring context after compaction. A reserved strip at the bottom marks the “Model output budget (reserved),” indicating space kept for generation. Below, a box labeled “LLM (reasoning engine)” notes that the model only sees what the harness allows through. A callout emphasizes that most tokens are not conversation and that the harness decides what stays.

Context management in agent harnesses: memory, files, and subagents

A version of this article originally appeared on X. Every agent harness runs into the same limit: the context window is too small for everything the model might want to remember….

13 minutes read
By Aparna Dhinakaran | 13 minutes read
Harness engineering

Subscribe to our resources and blogs

Subscribe