LLM applications generate vast amounts of logging data available for analysis. Even just a hundred traces can generate tens of thousands of span attributes. This makes it difficult to find patterns in your data, such as a cohort of questions where the chatbot response hallucinates its answers. You want to catch these before your users do and report a negative experience with your product.
Arize AI Agent Search helps engineers quickly identify patterns, uncover problematic traces, and debug applications with ease. No more sifting through endless rows of data or constructing complex queries by hand—just ask the AI agent directly for what you need. For example, you can:
- Find traces where users exhibit frustration.
- Identify hallucinated responses (where the LLM makes up information).
- Categorize data automatically and apply filters based on your needs.
- Cluster your data using LLMs.
Arize’s AI agent search is designed to streamline your data analysis workflow. Start by asking the AI agent to categorize your data, such as identifying different question types your chatbot receives, or different response types or problems.
You can also construct filter queries using natural language, such as asking for hallucinated responses. From there, you can dive deeper by asking the agent to reveal patterns in these hallucinations, helping you understand the root cause of the issue—whether it’s a misunderstanding of the input, bad retrieval, or LLM hallucination.
Improve your LLM application performance
After you’ve found your problematic traces, you can add them to a dataset to do experimentation and testing in Arize, preventing future performance regressions. We’ll be building agents into every part of the prompt iteration workflow, from improving your prompts, debugging your spans, and surfacing patterns in your data, all in the background so when you log into Arize, most of your work is already done.
The future of debugging and optimizing LLM applications is in agents. By harnessing Arize’s cutting-edge tools, you can focus more on building great products and less on data curation and debugging. Use Arize’s AI Agent Search to dive into your logging data, uncover patterns, and fix issues—before your users ever notice a problem. Happy building!
Sign up for a free Arize account
Or check out our open-source tool Phoenix
