The Evaluator

Agent Engineering Agent Evaluation Agents

3 production patterns for AI agents and how to evaluate each one

A local coding agent, an in-app customer assistant, and an AI SRE triaging production logs may all use the same model class—but not the same harness, eval plan, or rollout risk. Mastra CEO Sam Bhagwat breaks down the three production patterns and how to evaluate each one.

Agent Engineering Agents AI Engineering

What is a loop in AI engineering, anyway?

The AI engineering world is using “loop” to describe several different agent architectures. This post maps execution loops, task loops, product loops, system loops, and the human oversight loop that controls them.

Agent Evaluation Agent Observability Agents

Trace before you migrate: Measuring Kubernetes bottlenecks in AI agent sandboxes

Kubernetes is strong for long-lived services, but it is often a poor default for short-lived agent sandboxes. Trace sandbox creation, tool execution, eval latency, and full trajectory time before you migrate runtimes.

Agent Evaluation Agents AI Engineering

The agent is the user now: lessons from the founder of WorkOS

WorkOS founder Michael Grinich explains why the next era of AI engineering depends on the systems around agents: identity, permissions, evals, memory, and feedback loops that keep autonomous software from succeeding in the wrong ways.

AI Evaluation AI Observability LLM Evaluation

Evals in CI: How to write your LLM evals as tests with Arize Phoenix

If you’re struggling to get started with evals, you’re not alone. This post explains how to write LLM evals as ordinary tests in CI with Phoenix, pytest, and Vitest/Jest.

Agent Engineering Agents AI Engineering

Own the loop: A field guide to agent harnesses

As models become cheaper and more interchangeable, the durable advantage shifts to the agent harness: the loop, tools, memory, permissions, and workflow you can own and refine.

Agent Evaluation Agents AI Evaluation

How to evaluate AI agents, avoid reward hacking, and build better specs

Agent evals are repeatable tests that score whether AI agents completed a task correctly. Learn how to design rubrics, test suites, and trace-based evals that catch failures and prevent reward hacking.

Agent Evaluation Agents AI Evaluation

Model subsidies are ending. What do you do now?

Flat-rate AI plans are subsidizing agentic workloads. Learn why LLM inference costs are moving to metered pricing and how evals reveal cost per successful task.

AI Evaluation LLM As A Judge LLM Evals

AI evals are a data science problem: What most teams get wrong

Hamel Husain explains why the best AI teams treat LLM judges like classifiers, not dashboards.

Agent Observability Agents AI Observability

Trace and evaluate TrueFoundry AI Gateway traffic in Arize AX

Learn how TrueFoundry AI Gateway exports OpenTelemetry traces to Arize AX so teams can trace, evaluate, and monitor production LLM and agent traffic without embedding a vendor SDK in every service.