The Evaluator
Your go-to blog for insights on AI observability and evaluation.
Showing 121–130 of 458 posts (page 13 of 46)
Building the Data Flywheel for Smarter AI Systems with Arize AX and NVIDIA NeMo
Self-driving cars don’t get better by sitting in a lab. They improve by driving millions of miles, capturing edge cases, and feeding that data back into training. Tesla’s fleet generates real-world scenarios that become training data. The cars get smarter. The data gets better. The flywheel spins faster. This pattern of production data driving continuous…
Top LLM Tracing Tools
As of October 2025, 82% of enterprise leaders now rely on generative AI weekly according to a recent report from Wharton and GBK – with three in four seeing positive returns on AI investments. However, challenges remain in achieving scale and mainstream adoption of agents in production. Amongst multiple reasons, the lack of lifecycle management…
8 Top Prompt Testing & Optimization Tools (2026)
The best prompt testing and optimization tools for LLMs and multi-agent systems in 2026, compared including features, evals, and how to choose. If we were to give the year 2025 an AI-appropriate appellation, it would probably be ‘the year of the agents.’ And that’s only picked up steam in 2026. Building atop the startling advances…
Sign up for our newsletter, The Evaluator — and stay in the know with updates and new resources:
ServiceNow’s Tara Bogavelli on AgentArch: Benchmarking AI Agents for Enterprise Workflows
In our latest AI research paper reading, we hosted Tara Bogavelli, Machine Learning Engineer at ServiceNow, to discuss her team’s recent work on AgentArch, a new benchmark designed to evaluate and compare AI agent architectures across real-world enterprise workflows. As Tara described, the motivation behind AgentArch is to “move benchmarking closer to reality.” Instead of…
OpenAI’s Santosh Vempala Explains Why Language Models Hallucinate
In our latest AI research paper reading, we hosted Santosh Vempala, Professor at Georgia Tech and co-author of OpenAI’s paper, “Why Language Models Hallucinate.” This paper offers one of the clearest theoretical explanations to date of why AI models make up facts — connecting hallucinations to classical misclassification theory and statistical calibration. The conversation explored…
What Are the Top LLM Evaluation Tools?
AI agents and real-world applications of generative AI are debuting at an incredible clip this year, narrowing the time from AI research paper to industry application and propelling productivity growth across industries. From Cline in coding to AI agents deployed in real estate and construction, tremendous value is being created — but getting these agents…
Arize AI Achieves ISO/IEC 27001 Certification
Organizations running AI agents in production depend on Arize to operate securely at scale, logging over 1 trillion inferences and spans and 10 million evaluation runs monthly. Today, we’re proud to share that Arize AI has achieved ISO/IEC 27001 certification, underscoring an already-robust commitment to the highest standards of information security. Why This Matters ISO/IEC…
Keller Williams: Rise of the Agent Engineer
Austin, Texas-based Keller Williams Realty, LLC is the world’s largest real estate franchise by agent count. It has more than 1,000 market center offices and 161,000 affiliated agents. The franchise is No. 1 in units and sales volume in the U.S. Since 1983, the company has cultivated an agent-centric, technology-driven, and education-based culture that rewards…
Optimizing Coding Agent Rules (./clinerules) for Improved Accuracy
Coding agents have become the focal point of modern software development. Tools like Cursor, Claude Code, Codex, Cline, Windsurf, Devin, and many more are revolutionalizing how engineers write and ship code. A consistent pattern among the most capable agents is their reliance on a single, persistent system prompt rather than a chain of sub-prompts. Each…
Should I Use the Same LLM for My Eval as My Agent? Testing Self-Evaluation Bias
Thanks to Aparna Dhinakaran and Elizabeth Hutton for their contributions to this piece. When building and testing AI agents, one practical question that arises is whether to use the same model for both the agent’s reasoning and the evaluation of its outputs. Keeping the model consistent may simplify the setup and reduce costs, but it…
