The Evaluator
Your go-to blog for insights on AI observability and evaluation.
Showing 81–90 of 458 posts (page 9 of 46)
Arize AX Adds Native Support for NVIDIA NIM as AI Model Provider
We’re excited to announce that Arize AX now supports NVIDIA NIM as a native AI model provider. Enterprises running NIM-deployed models can now connect them directly to the Arize platform and access them from the playground, run experiments, online evaluations, and enable production monitoring — now as a dedicated first-class integration within Arize. What is…
How We Used Evals (and an AI Agent) to Iteratively Improve an AI Newsletter Generator
We love building little AI-powered tools that accelerate our workflows. One we built recently is a tool that takes our recent tweets and uses Claude to create a draft of our newsletter. It worked, sort of. The writing was good, but the details were a mess. So we pointed a coding agent at the problem:…
Arize Skills: Coding Agent Workflows for Traces, Evals, and Instrumentation
Two weeks ago we launched Alyx 2.0, the AI engineering agent inside Arize AX. Last week we launched the AX CLI, which made your trace data headless and agent-readable. Today we’re shipping the next piece: Arize Skills. The days of writing syntax are over We’re firmly in the agent era. You tell software what you…
Sign up for our newsletter, The Evaluator — and stay in the know with updates and new resources:
How to Build Planning Into Your Agent (The Architecture That Actually Works)
2025 was supposed to be the year of agents. And for the most part, it wasn’t. The industry was full of hype, demos looked incredible, but when you actually tried to get an agent to do something meaningful, it would fall apart. As we started digging into the few systems that did handle complex workflows…
From UI to Terminal: Bringing Alyx’s Superpowers Into Your Coding Agent
Last week we launched Alyx 2.0, the in-app AI engineering agent for Arize AX. Alyx replaced clicking through the UI with natural language intent. The AX CLI takes it a step further: making that same data machine readable so your coding agent can work with it directly. Here’s a quick demo of what that looks…
How to Evaluate Tool-Calling Agents
When you give an LLM access to tools, you introduce a new surface area for failure — and it breaks in two distinct ways: The model selects the wrong tool (or calls a tool when it should have answered directly). The model selects the right tool, but calls it incorrectly — wrong arguments, missing parameters,…
14 best AI agent observability tools in 2026: A practical comparison
Compare 14 AI agent observability tools for tracing, evaluations, OpenTelemetry, self-hosting, pricing, and production monitoring. Updated July 2026.
Add Observability to Your Open Agent Spec Agents with Arize Phoenix
Open Agent Specification lets you define an agent once and run it on any compatible runtime: LangGraph, WayFlow, CrewAI, and others. That portability solves a real problem in production AI systems. But it raises a follow-up question: once your agent is running, how do you know what it’s actually doing? Observability gives you the answer….
AI Agent Debugging: Four Lessons from Shipping Alyx to Production
Building AI systems that actually work in production is harder than it sounds. Not demo-ware, not “it worked once in a notebook.” Real systems that keep working after week two. We built Alyx, Arize’s agent for AX, and it broke in ways we didn’t expect. This post is about what broke, what surprised us, and…
Alyx 2.0: The AI Agent That Actually Plans
Two years ago, we started building Alyx with GPT-3.5, a vision, and honestly, no clear path forward. Agents were a buzzword. The models were rough. Tool calling was just emerging. But we had a hypothesis: the future wouldn’t be clicking through UIs or even just chatting with an assistant. You’d say what you want, and…
