09.25.2025: Repetitions 🔁

Available in Phoenix 11.38+

Since LLMs are probabilistic, their synthesis can differ even when the supplied prompts are exactly the same. This can make it challenging to determine if a particular change is warranted as a single execution cannot concretely tell you whether a given change improves or degrades your task. So what can you do when an execution can change from one run to the next? That's where repetitions come in.

Repetitions help you reduce uncertainty in systems prone to variability, notably more "agentic" systems.

feat: support repetitions by axiomofjoy · Pull Request #9657 · Arize-ai/phoenixGitHub

feat(playground): add repetitions to playground by axiomofjoy · Pull Request #9514 · Arize-ai/phoenixGitHub