> ## Documentation Index > Fetch the complete documentation index at: https://arize-ax.mintlify.dev/docs/llms.txt > Use this file to discover all available pages before exploring further. # Improve Your Agent > Use production failures to improve your prompt, then prove the fix works across your dataset In the previous guide, the groundedness [evaluator](/ax/evaluate/create-evaluators) revealed a pattern: the chatbot makes claims not in the policy documents - the system prompt says "be helpful" but doesn't enforce grounding. Rather than guessing at a fix and redeploying, start from a real failure, fix it in Playground using the exact inputs that went wrong, then validate across a full dataset before shipping. Arize AX skyserve-chatbot Traces view with Alyx assistant open, user request about groundedness-check failures this week, and Alyx task plan and progress This is **Part 3** of the Arize AX Get Started series. You should have completed the [Evaluations guide](/ax/get-started/get-started-evaluations) first, with [evaluation](/ax/evaluate/run-evals-on-traces#viewing-results) scores visible on your traces. ## Choose how you want to work Use [Arize Skills](/ax/agents/arize-skills) to have your coding agent run improvement workflows from your editor, [Alyx](/ax/alyx) for a conversational approach inside the Arize platform, the UI for a hands-on step-by-step experience, or **Code** to run programmatically. In each path, you'll build a dataset from failing traces, iterate on your prompt, and compare experiments before shipping. Use [Arize Skills](/ax/agents/arize-skills) with your coding agent to run the same workflow from your editor. The example prompts below are what you type to your agent — the skill loads automatically and handles the rest. Install the skills plugin and follow [Set up Arize with AI coding agents](/ax/set-up-with-ai-assistants) for authentication and CLI setup. ### Step 1: See evaluation results on your traces [`arize-trace`](https://github.com/Arize-ai/arize-skills/blob/main/skills/arize-trace/SKILL.md) Spans include labels once an [eval task](/ax/evaluate/run-evals-on-traces#create-a-task) has run; see [Viewing results](/ax/evaluate/run-evals-on-traces#viewing-results) in the tracing UI. For example, you might say: > Export spans from skyserve-chatbot where groundedness-check failed this week Terminal showing ax spans export command, export success message, summary of spans with low groundedness flagged as hallucinated, and a table of span and trace IDs with evaluator columns ### Step 2: Create a dataset [`arize-dataset`](https://github.com/Arize-ai/arize-skills/blob/main/skills/arize-dataset/SKILL.md) For example, you might say: > Create skyserve-test-cases from those failing traces Terminal: dataset skyserve-test-cases created from failing traces, with schema fields question, reference_text, original_output, trace and span IDs, and status counts ### Step 3: Improve the system prompt [`arize-prompt-optimization`](https://github.com/Arize-ai/arize-skills/blob/main/skills/arize-prompt-optimization/SKILL.md) For example, you might say: > Extract the system prompt from the failing skyserve-chatbot spans and generate an improved version. Use the groundedness-check eval labels and explanations as signal for what to fix. Improved SkyServe system prompt with grounding rules and notes referencing groundedness-check eval labels and span-level failures ### Step 4: Run both prompts as experiments [`arize-experiment`](https://github.com/Arize-ai/arize-skills/blob/main/skills/arize-experiment/SKILL.md) Reuse the same [evaluators](/ax/evaluate/create-evaluators) you trust in production; see [Run evals on experiments](/ax/evaluate/run-evals-on-experiments). For example, you might say: > Run both prompt versions (original and the updated one) against the dataset and compare groundedness scores. Experiment comparison table: skyserve-original-prompt at 4/5 groundedness versus skyserve-improved-prompt at 5/5 (100%) Use [Alyx](/ax/alyx) from [**Traces**](/ax/observe/tracing), the [**Prompt Playground**](/ax/prompts/prompt-playground), [**Datasets**](/ax/develop/datasets-and-experiments), or [**Experiments**](/ax/develop/datasets-and-experiments/compare-experiments) to run the same workflow in conversation. Then, follow the flow below. Arize AX skyserve-chatbot Traces view with Alyx assistant open, user request about groundedness-check failures this week, and Alyx task plan and progress ### Step 1: See evaluation results on your traces Ask about traces that already have [evaluation](/ax/evaluate/run-evals-on-traces#viewing-results) columns from your [eval tasks](/ax/evaluate/run-evals-on-traces#create-a-task). For example, you might say: > Show me traces where groundedness-check failed this week and explain what went wrong ### Step 2: Create a dataset For example, you might say: > Create skyserve-test-cases from those failing traces ### Step 3: Improve the system prompt Grounding failures should line up with [evaluator](/ax/evaluate/create-evaluators) labels from [online evals on traces](/ax/evaluate/run-evals-on-traces). For example, you might say: > Extract the system prompt from that trace and suggest an improved version based on the groundedness failures ### Step 4: Run both prompts as experiments Compare runs with the same [evaluators](/ax/evaluate/create-evaluators) you use in production; see [Run evals on experiments](/ax/evaluate/run-evals-on-experiments). For example, you might say: > Run both prompt versions (original and the updated one) against the dataset and compare groundedness scores. Playground compare view: versions A and B on skyserve-test-cases with groundedness scores and per-row grounded versus ungrounded labels ### Step 5: Save to Prompt Hub When you are happy with the improved prompt, ask Alyx to save it to [Prompt Hub](/ax/prompts/prompt-hub) so you get a named template, version history, and rollbacks - the same outcome as clicking **Save to Prompt Hub** in the Playground UI. For example, you might say: > Save the improved system prompt from this Playground to Prompt Hub as skyserve-support. Use a version description like: added explicit grounding rules so the model refuses when the policy docs do not support an answer. Follow these steps in the Arize AX UI: find failing traces, replay them in the [Prompt Playground](/ax/prompts/prompt-playground), tighten your prompt, build a dataset, run [experiments](/ax/develop/datasets-and-experiments/compare-experiments), compare [evals](/ax/evaluate/run-evals-on-experiments), and save to [Prompt Hub](/ax/prompts/prompt-hub). ### Step 1: See evaluation results on your traces Go to your **skyserve-chatbot** project and filter traces by the **groundedness-check** [evaluation](/ax/evaluate/run-evals-on-traces#viewing-results) score. Find a trace that failed — one where the chatbot made up information not in the policy documents — and click in to see what went wrong. Traces filtered by groundedness evaluation showing hallucinated traces ### Step 2: Replay in Prompt Playground Click **Open in Playground** on the span. AX automatically populates the system prompt, user message, and model settings that produced the bad answer — no manual setup needed. Trace detail for an LLM ChatCompletion span showing trace tree, span evaluations, Open in Playground, and Input Output tab with model and system prompt
Prompt Playground auto-populated from a trace with system prompt, user message, and model ### Step 3: Improve the system prompt The original is too loose: ``` You are SkyServe Airlines' customer service assistant. Answer the customer's question based on the provided policy documents. Be friendly and helpful. ``` Tighten it with explicit grounding rules — for example, require that every claim reference a specific policy, and instruct the model to say "I don't have that information" rather than guess. Click **Run** to confirm the response improves. Playground with improved system prompt and new grounded response ### Step 4: Create a dataset and run experiments A few spot-checks aren't enough. Create a dataset of representative test cases (common questions, edge cases, known failures) and run both prompt versions against it as experiments — one baseline, one improved. In **Datasets**, add examples (upload a CSV or build from traces) and open the dataset in Playground. Run your original prompt as the baseline experiment, then run your improved prompt on the same inputs. New Dataset dialog showing CSV upload with preview of test cases
Experiments tab showing baseline-original-prompt experiment
Playground with improved prompt and dataset ready to run ### Step 5: Evaluate and compare Add your **groundedness-check** [evaluator](/ax/evaluate/create-evaluators) to both experiments (the same one you created in the [Evaluations guide](/ax/get-started/get-started-evaluations)) and use **Compare** mode to view results side by side. You can add a Helpfulness [evaluator](/ax/evaluate/create-evaluators) from the templates to check that answers stay useful. You should see groundedness improve while helpfulness stays flat. If you see a regression, iterate in Playground. Add Evaluator flow showing available evaluators from the hub
Compare Experiments view showing two experiments side by side ### Step 6: Save to Prompt Hub Once satisfied, click **Save to Prompt Hub**, name it `skyserve-support`, and add a version description. Your prompt is now versioned — your team can see the full history, compare versions, and roll back if needed. Save to Prompt Hub dialog with name, description, and version description
Prompt Hub showing skyserve-support version history and prompt template Run this workflow from the [Python SDK](/api-clients/python/overview), [TypeScript SDK](/api-clients/typescript/version-1/overview), or [`ax` CLI](/api-clients/cli/overview). Some features are in alpha or beta - please check individual reference pages for details. | Step | Python SDK | TypeScript SDK | CLI | | ---------------------------- | ------------------------------------------------------------------ | ---------------------------------------------------------------------- | ------------------------------------ | | Filter spans by eval result | [Link](/api-clients/python/version-8/client-resources/spans) | [Link](/api-clients/typescript/version-1/client-resources/spans) | [Link](/api-clients/cli/spans) | | Create a dataset from traces | [Link](/api-clients/python/version-8/client-resources/datasets) | [Link](/api-clients/typescript/version-1/client-resources/datasets) | [Link](/api-clients/cli/datasets) | | Manage prompts | [Link](/api-clients/python/version-8/client-resources/prompts) | [Link](/api-clients/typescript/version-1/client-resources/prompts) | [Link](/api-clients/cli/prompts) | | Run experiments | [Link](/api-clients/python/version-8/client-resources/experiments) | [Link](/api-clients/typescript/version-1/client-resources/experiments) | [Link](/api-clients/cli/experiments) | ## Congratulations! You've completed the full improvement loop: 1. Traced your app to see what's happening inside it. 2. Evaluated responses automatically to measure quality. 3. Improved your prompt using real failure data in the Playground. 4. Proved the improvement works across a representative dataset with experiments. You now have a repeatable, data-driven process for improving your LLM application. No more guessing, no more hoping - you can measure quality and demonstrate improvement. **Next up:** Deepen your tracing foundation so your improvement loop stays grounded in complete, high-quality telemetry.