Alyx: Eval & RAG Analysis

Use Alyx to Understand Your Evaluations

Rather than manually combing through your evaluation results, ✨Alyx assesses and summarizes your evaluation metrics, offering targeted suggestions for improving your LLM application. These insights help you identify shortcomings in your application and make the necessary adjustments to enhance performance.

1. Summarize Evaluation Metrics

• Suggested Prompt: "Summarize my <eval_name> eval"

• Use When: You want to understand the performance of your LLM application.

• Description: Assesses and summarizes your evaluation metrics, offering targeted suggestions for enhancing your LLM application.

2. RAG Analysis

Debug retrieval directly from trace details or in the main chat

• Suggested Prompt: "Debug retrieval step of RAG app for <trace_id>"

• Use When: You need to refine your retrieval process.

• Description: Evaluate and refine your retrieval process. Analyzes responses to ensure relevance and accuracy, and provides improvement strategies.