The Evaluator

Prompt Engineering Research

How to Make Your AI App Feel Magical: Prompt Caching

Credit to Harrison Chu for the research behind this post A key ingredient to making your AI app feel “magical” is speed—snappy feedback enhances user experience significantly. Companies like Cursor achieve this by “pre-warming” their cache, adding relevant information as soon as users start interacting. For those looking to boost AI app performance, prompt caching…

AI Evaluation AI Product Quality LLM Evals

Arize, Vertex AI API: Evaluation Workflows to Accelerate Generative App Development and AI ROI

Written in collaboration with Christian Williams, Principal Architect AI/ML, Google Cloud.  In the rapidly evolving landscape of artificial intelligence, enterprise AI engineering teams must constantly seek cutting-edge solutions to drive innovation, enhance productivity, and maintain a competitive edge. In leveraging an AI observability and evaluation platform like Arize AI with the advanced capabilities of Google’s…

Agent Engineering Agents AI Engineering

Swarm: OpenAI’s Experimental Approach to Multi-Agent Systems

As multi-agent systems grow in importance for fields ranging from customer support to autonomous decision-making, OpenAI has introduced Swarm, an experimental framework that simplifies the process of building and managing these systems. Swarm, a lightweight Python library, is designed for educational purposes, stripping away complex abstractions to reveal the foundational concepts of multi-agent architectures. In…

Agent Observability AI Observability LLM Observability

Zero to a Million: Instrumenting LLMs with OTEL

Thanks to Roger Yang, Xander Song, and John Gilhuly for their contributions to this piece. A few months ago, we hit a significant milestone: our OTEL LLM instrumentation surpassed one million monthly downloads. This journey has been challenging but rewarding. We, along with other key players in the industry, are paving the way for observability…

AI Evaluation AI Observability LLM Evaluation

Arize Release Notes: Test Tasks, Filter Experiments, and More

Welcome to our regular update on new releases, enhancements, and changes. What’s New Run Task Once Users now have the option to to test a task, such as online eval, by running it once on existing data, or apply evaluation labels to older traces. To use the feature, simply select “Run once on historical data”…

AI Evaluation LLM Evals LLM Evaluation

Techniques for Self-Improving LLM Evals

LLM evaluations have become a great tool for benchmarking performance - and they’re particularly useful for times where measuring the quality of the output is complicated, like in summarization or classification, for instance. Teams sometimes use LLM-assisted evals to help with things like ranking or suggesting improvements for the evaluation process. LLM-as-a-judge is a popular way AI…

Agent Engineering Agent Evaluation Agent Observability

Tracing and Evaluating LangGraph Agents

LangGraph is a powerful library designed for building stateful, multi-actor applications within large language models (LLMs). In this post, we’ll discuss how LangGraph’s traces can be ingested into Arize, and how to leverage LLMs as a judge to evaluate LangGraph agent performance. What is LangGraph? LangGraph is a flexible, low-level framework that allows developers to…

Agent Engineering AI Engineering Open Source

OpenAI Swarm Explained: Multi-Agent Orchestration

Last week, OpenAI introduced Swarm, the latest addition to the rapidly evolving multi-agent framework space. Swarm joins the ranks of frameworks like CrewAI and Autogen, pushing the boundaries of how agents work collaboratively in complex AI systems. Although Swarm is still in its early stages, comparing it to established players like CrewAI and Autogen highlights…

AI Engineering

Google’s NotebookLM and the Future of AI-Generated Audio

In this paper read, Aman Khan and Harrison Chu explore NotebookLM’s unique features, including its ability to generate realistic-sounding podcast episodes from text. Khan reviews NotebookLM from the product side, and they discuss the technical underpinnings of the product from the outside looking in. Chu provides an overview of the SoundStorm model that’s suspected to…

Agent Engineering Agent Observability AI Engineering

The Role of OpenTelemetry (OTEL) in LLM Observability

If you’ve ever tried developing–or harder yet, productionizing–an LLM application, you know that getting things to work as intended is not as easy as you think. Excluding demos on X, many folks struggle to build a quality LLM application into a viable product. For the first time, many of us are designing systems that have…