The Evaluator
Your go-to blog for insights on AI observability and evaluation.
Showing 1–10 of 464 posts (page 1 of 47)
How Uber evaluates AI agents at production scale
A background comment about pizza exposed a failure that Uber’s offline evaluations had missed. The incident helped reveal what production AI agent evaluation actually requires: automatic tracing, living datasets, shared ownership, and a direct connection to product decisions.
Arize and Dynatrace: Making the World’s AI Work
Today we are announcing the signing of a definitive agreement for the acquisition of Arize by Dynatrace to accelerate our mission to make the world’s AI work.
AI agent guardrails vs. evals: How to build more reliable agent systems
Guardrails constrain what an agent can do in code; evals judge whether it performed well. Learn how both layers—and the harness around them—make long-running AI agents reliable.
Sign up for our newsletter, The Evaluator — and stay in the know with updates and new resources:
Evaluation-driven development: How to move AI agents from pilot to production
Learn how evaluation-driven development, agent harnesses, AI observability, guardrails, and cost-per-outcome metrics move AI agents from pilot to production.
Crew Studio launches with native Arize AX tracing and evaluation
Through a native Arize AX integration, teams can send traces from Crew Studio to Arize from the first run without custom instrumentation—then inspect behavior, evaluate quality, and test fixes before redeploying.
You chose the best model. Why is your agent still failing?
Public benchmarks can show how a model performs in general. Production reliability depends on the context and harness around it, which only your team can evaluate against its own data, workflows, and users.
Arize AX adds native support for OpenTelemetry GenAI semantic conventions
Arize AX now normalizes OpenTelemetry GenAI semantic conventions into first-class AI traces, unlocking evaluations, token and cost visibility, and easier debugging.
Demystifying the EU AI Act for AI product and engineering teams
An engineering guide to turning EU AI Act principles into traces, evaluations, annotations, and release evidence product and engineering teams can actually demonstrate.
How cheap models changed multi-agent economics
Orchestrator-executor just became the smart default for production agents: an expensive model plans, cheap models execute, and cost per completed task decides the roster.
AI agent observability: Why production systems need a reasoning layer
Traditional APM can collect every span and still leave developers guessing about intent, causality, and drift. As agents multiply, the observability stack must learn to interpret the systems it watches.
