The Definitive LLM Observability Checklist

According to a recent survey, only 30.1% of teams deploying LLMs have implemented observability despite large majorities wanting better debugging workflows and ways to tackle hallucinations, toxicity, and other issues. As teams play catch-up, what should they look for when assessing an LLM observability platform? Informed by experience working with hundreds of practitioners across dozens of large enterprises and technology companies with LLM apps in production, this checklist covers essential elements to consider when evaluating an LLM observability provider.

Dive into details on essentials like:
  • LLM System Evaluations
  • LLM Traces and Spans
  • Prompt Engineering
  • Retrieval Augmented Generation
  • Fine-Tuning
  • Embeddings Analysis
  • Platform Support

Read the Checklist

About the author

Aparna Dhinakaran
Co-founder & Chief Product Officer

Aparna Dhinakaran is the Co-Founder and Chief Product Officer at Arize AI, a pioneer and early leader in machine learning (ML) observability. A frequent speaker at top conferences and thought leader in the space, Dhinakaran was recently named to the Forbes 30 Under 30. Before Arize, Dhinakaran was an ML engineer and leader at Uber, Apple, and TubeMogul (acquired by Adobe). During her time at Uber, she built several core ML Infrastructure platforms, including Michealangelo. She has a bachelor’s from Berkeley's Electrical Engineering and Computer Science program, where she published research with Berkeley's AI Research group. She is on a leave of absence from the Computer Vision Ph.D. program at Cornell University.

Get ML observability in minutes.

Get Started