Prompt Optimization (Beta)

Arize offers multiple ways to optimize your prompts for better LLM performance.

Why Optimize your Prompts?

Modern LLMs are already highly capable — but how you guide them matters just as much as the model itself. A strong prompt can dramatically boost reasoning, consistency, and accuracy without retraining. Even top systems like Claude Sonnet 3-7 rely on massive, hand-tuned prompts (about 24k tokens) to define their reliability and depth.

Most teams can’t afford that level of manual engineering — which is why data-driven prompt optimization is so powerful. By using evaluations, natural language feedback, and production traces, prompts can be refined automatically based on real performance data. In our tests, this approach improved coding accuracy on SWE-Bench by 10–15% and reasoning scores on Big Bench Hard by up to 10%. Prompt optimization lets every team shape LLM behavior with the same rigor as top AI labs — but through automation and data, not guesswork.

Prompt Optimization Methods in Arize AX

Prompt Optimization via UI

• Create and manage Prompt Optimization tasks directly in the Arize interface — no code required.

• Run feedback-driven optimization loops, compare prompt versions side-by-side, and promote the best one to production in just a few clicks.

Prompt Learning via SDK

• Automate prompt optimization programmatically using the Prompt Learning SDK, which iteratively refines prompts based on evaluation feedback and annotations.

• Supports advanced features, like built-in systems to run evaluators, and more flexibility to run multiple loops or define train/test splits

Optimize prompts with Alyx (Copilot)

• Collaborate with Alyx, your conversational copilot, to refine prompts for clarity, tone, or factuality using natural language guidance.

• Simply describe your goal — for example, “Optimize my prompt to reduce verbosity” — and Alyx will generate an improved version instantly.