The Evaluator
Your go-to blog for insights on AI observability and evaluation.
Showing 221–230 of 458 posts (page 23 of 46)
Arize Release Notes: Embeddings Tracing, Experiments Details, and More.
Welcome to our regular update on new releases, enhancements, and changes. What’s New Embeddings Tracing With Embeddings Tracing, you can effortlessly select embedding spans and dive straight into the UMAP visualizer, simplifying troubleshooting for your genAI applications. How this works: Users can now select embedding spans and go to the embedding visualizer (also available on…
Building AI Assistants with Vectara-agentic and Arize
Thanks to the Vectara team for contributing this post! Introduction Retrieval-Augmented Generation (RAG) is a framework that enhances the capabilities of large language models (LLMs) by integrating external information retrieval systems to provide more relevant and factual responses. In a standard RAG setup, the model retrieves documents from a knowledge base, and feeds those documents…
Best Practices for Selecting the Right Model for LLM-as-a-Judge Evaluations
When building and scaling LLM-based applications, ensuring model performance is critical. One powerful method for evaluating that performance is using an LLM as a judge. This allows you to systematically assess your model’s output and detect potential issues like hallucinations, incorrect responses, or inconsistencies that can arise as the model starts handling real-world data. In…
Sign up for our newsletter, The Evaluator — and stay in the know with updates and new resources:
Arize AI + MongoDB: Leveraging Agent Evaluation and Memory to Build Robust Agentic Systems
In the evolving landscape of artificial intelligence, agentic systems—autonomous agents capable of making decisions and learning from feedback loops in their environment—are becoming increasingly sophisticated. At the same time, as retrieval augmented generation (RAG) applications become more complex, a critical component of these systems is memory. AI agents depend on memory to perform effectively, adapt…
Exploring OpenAI’s o1-preview and o1-mini
OpenAI recently released its o1-preview, which they claim outperforms GPT-4o on a number of benchmarks. These models are designed to think more before answering and handle complex tasks better than their other models, especially science and math questions. We take a closer look at their latest crop of o1 models (including o1-mini), and we also…
Breaking Down Reflection Tuning: Enhancing LLM Performance with Self-Learning
A recent announcement on X boasted a tuned model with pretty outstanding performance, and claimed these results were achieved through reflection tuning. However, people were unable to reproduce the results. We dive into some recent drama in the AI community as a jumping off point for a discussion about Reflection 70B. In 2023, there was…
Arize Release Notes: AI Search V2, Copilot Updates, and More
Welcome to our regular update on new releases, enhancements, and changes. What’s New AI Search V2 We’re excited to announce the release of AI Search V2, packed with new features and improvements designed to enhance the user experience. Here’s what’s new: Column Search (Improved) The original AI search skill now offers refined semantic search capabilities…
Tracing a Groq Application
Special thanks to Duncan McKinnon for his contributions to this post Tracing Groq Applications with Arize If you’re working with LLMs and using the Groq package to make your model calls, capturing telemetry data is crucial for debugging, monitoring, and improving your system’s performance. In this post, we’ll walk through a quick guide on how…
Composable Interventions for Language Models
Introduction We’re excited to be joined by Kyle O’Brien, Applied Scientist at Microsoft, to discuss his most recent paper, Composable Interventions for Language Models. Kyle and his team present a new framework, composable interventions, that allows for the study of multiple interventions applied sequentially to the same language model. The discussion will cover their key…
Arize Release Notes: Sep 5, 2024
Welcome to our regular update on new releases, enhancements, and changes. What’s New Annotations (Beta) Annotations are custom labels that can be added to traces. Use annotations to: manually label data, categorize spans or traces, curate a dataset for experimentation, log human feedback. Reach out to your account rep to get access. Learn more about…
