Skip to main content
Laurie Voss demonstrates an end-to-end evaluation workflow using a financial analysis agent. The workshop starts with traces and failure analysis, then builds code evaluations, LLM-as-a-judge evaluations, and a custom rubric. It concludes with experiments for checking whether a prompt change improves results.

​
What you’ll learn

  • Use traces to understand agent behavior before writing evaluations.
  • Categorize failures by root cause.
  • Build code-based, LLM-as-a-judge, and rubric-based evaluations.
  • Use datasets and experiments to verify changes instead of relying on spot checks.
Watch the workshop on YouTube.