The Evaluator

Agent Engineering Agent Evaluation Agent Observability

How to build a better agent harness with traces and evals

Agents are easy to prototype and hard to improve. A repeatable loop of traces, evals, failed-span inspection, and targeted harness changes makes agent behavior easier to debug and improve.

Agent Evaluation Agent Observability AI Evaluation

From production traces to better AI agents: Automating the LLMOps feedback loop

Production AI traces are the raw material for better evals, prompts, datasets, and fine-tuned models. This post shows how the Arize AX Airflow Provider turns that feedback loop into scheduled, monitored LLMOps pipelines.

AI Evaluation LLM Evals LLM Evaluation

How to ship a local LLM that matches frontier LLMs with evals and prompt engineering

Most production AI features don’t need a frontier model. Here’s how capability evals and prompt engineering can help ship a local SLM that matches frontier-model quality with lower latency and cost.

AI Evaluation AI Observability LLM As A Judge

How to build LLM-as-a-Judge evaluators that hold up in production

Learn how to design, calibrate, and run LLM-as-a-judge evaluators with fixed labels, human agreement checks, trace context, and Phoenix Evals.

Agent Engineering Agent Evaluation AI Engineering

What we learned testing 7 models under the same agent harness

Model swaps look like configuration changes, but they behave more like product migrations. A new model may be cheaper, faster, easier to get capacity for, or stronger on public benchmarks. The API call may barely change. From the outside, swapping models can look as simple as changing a model name in a config file. But…

Agent Engineering AI Engineering

Building a self-improving agent on a context graph of human disagreement

You can build a measurably better agent from data you already have, without retraining a thing. The data is what your experienced humans do when they correct the AI. Capture those domain-knowledge based corrections as a context graph, mine them for patterns, and the agent steadily matches what the humans actually do. Every AI agent…

Agent Engineering Agent Evaluation Agent Observability

Coding agent tracing and evaluation: An open source tool to improve AI coding workflows

Announcing coding harness tracing for observing, evaluating, and improving coding agent workflows across Claude Code, Cursor, Codex, GitHub Copilot, and Gemini CLI.

Agent Engineering Agent Evaluation Agent Observability

How we use Alyx to build Alyx: How to build an AI agent feedback loop

How Arize uses Alyx to debug Alyx: searching dense traces, aggregating failures, triaging dogfooding issues, and closing the AI engineering feedback loop.

AI Evaluation

Models got an order of magnitude better at following instructions in one year

A year ago, frontier models started losing track of instructions somewhere around 200–300 simultaneous constraints. With 2026 models, that ceiling is closer to 2,000 — an order-of-magnitude jump. We re-ran IFScale to see how, and how each model fails.

Agent Evaluation Agent Observability AI Evaluation

From observability to context: What’s next for Arize Phoenix

As agents start changing software, they need a way to verify their work that includes traces, evals, feedback, and APIs. This is where Phoenix goes next — not the next release, but what this product becomes.