A golden dataset is one that contains trusted inputs and ideal outputs. These are typically hand-labeled by humans (often with domain expertise) and serve as a benchmark for model output quality.
In AI development, building an evaluation from the ground up requires iteration and testing and golden datasets are often instrumental. This tutorial walks through how to create a benchmark dataset with annotations, then develop a custom LLM evaluator. We refine the evaluator against the golden dataset to ensure it meets quality standards, highlighting practical techniques for improving evaluator accuracy over time.
📓 Notebook
📊 Learn more about datasets
- Phoenix OSS
- Pricing
- Docs
-
Learn
Join the Arize ML Observability Community
Subscribe to the Arize blog
Subscribe to get the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff.
*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our privacy policy.
Contact us
Like what you see? Let’s chat. Fill out this form and we will be in contact with you soon!