Nancy Chauhan
-
Open SourceArize Phoenix now updates model prices automatically
Keep built-in model prices up to date with automatic catalog updates between Phoenix releases. Nancy Chauhan November 2, 2026 4 min read -
Agent EvaluationA web page talked our agent into leaking a stranger’s data. Here’s how we caught it.
A web page told our support agent to look up a different customer, and the agent complied on 28 of 100 runs. Here's how we found the leak… Nancy Chauhan October 26, 2026 15 min read -
Agent EvaluationAgent tool selection: The tool your agent almost called
Decision spans in Arize Phoenix record a tool router’s choice, alternative probabilities, and confidence. Here’s how we evaluated those decisions and used the results to set a fallback… Nancy Chauhan October 19, 2026 14 min read -
Agent EvaluationHow to test agent tool descriptions with Harbor and Phoenix
Build a reproducible benchmark, compare tool use across agents, and test whether a description change reduces unnecessary work while preserving outcomes. Nancy Chauhan October 12, 2026 11 min read -
Agent EngineeringPrompt caching benchmark: high cache reuse doesn’t always mean lower cost
We benchmarked prompt caching across DeepSeek, GLM, GPT, and Claude using Harbor evals and Phoenix traces, so we could compare cache reuse, estimated cost, and latency on the… Nancy Chauhan October 2, 2026 7 min read -
Agent EngineeringAI agent regression testing with Agent Experiments in Arize AX
A cancellation-policy fix raised action safety and dropped average task completion from 0.89 to 0.72. This walkthrough shows how to regression-test agent changes with Agent Experiments in Arize… Nancy Chauhan Fuad Ali September 14, 2026 9 min read -
AI EngineeringHow to reduce LLM costs without sacrificing quality
Trace where your AI budget goes, identify the requests that do not justify their spend, and validate lower-cost alternatives with Arize AX. Nancy Chauhan August 31, 2026 11 min read -
Agent EngineeringHow Signal found two hidden retry loops in our production agent Alyx
We ran Signal on Alyx, the AI engineering agent built into Arize AX. It surfaced a duplicate task-state loop and a 43-call dataset retry that appeared as valid… Nancy Chauhan August 27, 2026 8 min read -
Agent EvaluationArize Phoenix has a built-in MCP server that lets your agents query traces with SQL
Read-only SQL and code mode let coding agents answer questions across your traces without paging thousands of spans through model context. Nancy Chauhan Roger Yang Mora Vigo Malusardi August 27, 2026 10 min read -
AI EngineeringHow to debug production AI agents with Signal in Arize AX
Learn how Arize Signal turns production traces into ranked issues, proposed fixes, regression datasets, and reviewable pull requests for AI agents. Nancy Chauhan August 4, 2026 16 min read
Don’t ship vibes.
Arize gives AI teams observability and evals to understand and improve agent performance.