support-classifier prompt. Version 3 adds a new instruction. Version 4 also changes the model and its parameters. At the end, you know how much each change improves accuracy.
Follow along with code: This guide has a companion notebook with runnable code examples. Find it here, and go to Part 3: Compare Prompt Versions.
Build Two New Prompt Versions
In Test Prompts at Scale, the prompt had 53% accuracy. Use the results of that analysis to create the new versions.Edit Prompt Template (Version 3)
The prompt template is the text that goes to the LLM. In Test Prompts at Scale, 30 of 71 errors had thebroad_vs_specific error type. This is the instruction that you wrote to fix these errors:
- UI
- Python
- TypeScript
Edit Prompt Parameters (Version 4)
A Phoenix prompt also stores the model and its invocation parameters. For more about what a prompt contains, see Prompts Concepts. Create Version 4 with these changes to the model and parameters:- UI
- Python
- TypeScript
Edit prompt parameters
Compare Prompt Versions
Run an experiment with each new version on the dataset. Then compare the accuracy of the versions.- Python
- TypeScript
In the Phoenix UI, open the
support-classifier prompt. Copy the version IDs of Version 3 and Version 4.
Summary
You created two prompt versions and measured each one.- New instruction only: Accuracy increased from 53% to 61%.
- New instruction, new parameters, and new model: Accuracy increased to 74%.

