AI Observability and Evaluation Platform

“We adopted Phoenix due to its excellent documentation and support and well designed ability to integrate quickly into our existing prototyping workflows. Arize has also nurtured an active community of LLMOps learners, professionals, and advocates that I’ve personally found very helpful to (try to) stay on top of new developments.”

“LLM applications are complex. To optimize them for speed, cost, or accuracy, you need to understand their internal state. Each step of the response generation process needs to be monitored, evaluated, and tuned. Phoenix lets us evaluate whether a retrieved chunk contains an answer to a query.”

“Arize observability is pretty awesome!”

“Arize offers an AI observability and LLM evaluation platform that helps AI developers and data scientists monitor, troubleshoot, and evaluate LLM models. This offering is critical to observe and evaluate applications for performance improvements in the build-learn-improve development loop..”

“Our big use case in Arize was around observability and being able to show the value that our AIs bring to the business by reporting outcome statistics into Arize so even non-technical folks can see those dashboards — hey, that model has made us this much money this year, or this client isn’t doing as well there — and get those insights without having to ask an engineer to dig deep in the data.”

“The US Navy relies on machine learning models to support underwater target threat detection by unmanned underwater vehicles. To ensure successful deployment of this technology, AI infrastructure is required to continuously monitor and improve model performance to ensure the systems remain effective. After a competitive evaluation process, Defense Innovation Unit (DIU) and the U.S. Navy awarded five prototype agreements in the fall of 2022 to Arize AI [and others] …as part of Project Automatic Target Recognition using Machine Learning Operations (MLOps) for Maritime Operations, nicknamed Project AMMO).”

“You have to define it not only for your models but also for your products…There are LLM metrics, but also product metrics. How do you combine the two to see where things are failing? That’s where Arize has been a fabulous partner for us to figure out and create that traceability.”

“For exploration and visualization, Arize is a really good tool.”

“We are constantly iterating on our production ranking model to improve activity relevance and personalization for our users’ unique preferences. As we launch A/B tests, Arize gives us the ability to break the performance further down into different data segments and highlight which features contribute to the model’s predictive performance the most. This gives us a broad overview of our ranking model’s overall performance at any time and allows us to identify areas of improvement, compare different datasets, and examine problematic slices.”