This tutorial will use Phoenix to compare the performance of different prompt optimization techniques.
Google Colab
colab.research.google.com
You’ll start by creating an experiment in Phoenix that can house the results of each of your resulting prompts. Next you’ll use a series of prompt optimization techniques to improve the performance of a jailbreak classification task. Each technique will be applied to the same base prompt, and the results will be compared using Phoenix.The techniques you’ll use are:
Few Shot Examples: Adding a few examples to the prompt to help the model understand the task.
Meta Prompting: Prompting a model to generate a better prompt based on previous inputs, outputs, and expected outputs.
Prompt Gradients: Using the gradient of the prompt to optimize individual components of the prompt using embeddings.
DSPy Prompt Tuning: Using DSPy, an automated prompt tuning library, to optimize the prompt.
This tutorial requires and OpenAI API key.
Let’s get started!
Setup Dependencies & Keys
!pip install -q "arize-phoenix>=8.0.0" datasets
Point your code at the Phoenix instance you started. The endpoint below is the default for a local phoenix serve; for a deployment running elsewhere, use its hostname instead.
import osos.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"# Only if the deployment has authentication enabled# os.environ["PHOENIX_API_KEY"] = "your-api-key"
Then provide your OpenAI key, which this tutorial uses for the LLM calls:
import osfrom getpass import getpassif not os.environ.get("OPENAI_API_KEY"): os.environ["OPENAI_API_KEY"] = getpass("Enter your OpenAI API key: ")
Load Dataset into Phoenix
Since we’ll be running a series of experiments, we’ll need a dataset of test cases that we can run each time. This dataset will be used to test the performance of each prompt optimization technique.
from datasets import load_datasetds = load_dataset("jackhhao/jailbreak-classification")["train"]ds = ds.to_pandas().sample(50)ds.head()
import uuidfrom phoenix.client import Clientunique_id = uuid.uuid4()# Upload the dataset to Phoenixpx_client = Client()dataset = px_client.datasets.create_dataset( dataframe=ds, input_keys=["prompt"], output_keys=["type"], name=f"jailbreak-classification-{unique_id}",)
Next, you can define a base template for the prompt. We’ll also save this template to Phoenix, so it can be tracked, versioned, and reused across experiments.
from openai import OpenAIfrom openai.types.chat.completion_create_params import CompletionCreateParamsBasefrom phoenix.client.types import PromptVersionparams = CompletionCreateParamsBase( model="gpt-3.5-turbo", temperature=0, messages=[ { "role": "system", "content": "You are an evaluator that decides whether a given prompt is a jailbreak risk. Only output 'benign' or 'jailbreak', no other words.", }, {"role": "user", "content": "{{prompt}}"}, ],)prompt_identifier = "jailbreak-classification"prompt = px_client.prompts.create( name=prompt_identifier, prompt_description="A prompt for classifying whether a given prompt is a jailbreak risk.", version=PromptVersion.from_openai(params),)
You should now see that prompt in Phoenix:
Next you’ll need a task and evaluator for the experiment. A task is a function that will be run across each example in the dataset. The task is also the piece of your code that you’ll change between each run of the experiment. To start off, the task is simply a call to GPT 3.5 Turbo with a basic prompt.You’ll also need an evaluator that will be used to test the performance of the task. The evaluator will be run across each example in the dataset after the task has been run. Here, because you have ground truth labels, you can use a simple function to check if the output of the task matches the expected output.
You can also instrument your code to send all models calls to Phoenix. This isn’t necessary for the experiment to run, but it does mean all your experiment task runs will be tracked in Phoenix. The overall experiment score and evaluator runs will be tracked regardless of whether you instrument your code or not.
from openinference.instrumentation.openai import OpenAIInstrumentorfrom phoenix.otel import registertracer_provider = register(project_name="prompt-optimization")OpenAIInstrumentor().instrument(tracer_provider=tracer_provider)
Now you can run the initial experiment. This will be the base prompt that you’ll be optimizing.
You should now see the initial experiment results in Phoenix:
1st experiment results
Prompt Optimization Technique #1: Few Shot Examples
One common prompt optimization technique is to use few shot examples to guide the model’s behavior.Here you can add few shot examples to the prompt to help improve performance. Conveniently, the dataset you uploaded in the last step contains a test set that you can use for this purpose.
from datasets import load_datasetds_test = load_dataset("jackhhao/jailbreak-classification")[ "test"] # this time, load in the test set instead of the training setfew_shot_examples = ds_test.to_pandas().sample(10)
Define a new prompt that includes the few shot examples. Prompts in Phoenix are automatically versioned, so saving the prompt with the same name will create a new version that can be used.
few_shot_template = """You are an evaluator that decides whether a given prompt is a jailbreak risk. Only output "benign" or "jailbreak", no other words.Here are some examples of prompts and responses:{examples}"""params = CompletionCreateParamsBase( model="gpt-3.5-turbo", temperature=0, messages=[ {"role": "system", "content": few_shot_template.format(examples=few_shot_examples)}, {"role": "user", "content": "{{prompt}}"}, ],)few_shot_prompt = PhoenixClient().prompts.create( name=prompt_identifier, prompt_description="Few shot prompt", version=PromptVersion.from_openai(params),)
You’ll notice you now have a new version of the prompt in Phoenix:
Meta prompting involves prompting a model to generate a better prompt, based on previous inputs, outputs, and expected outputs.The experiment from round 1 serves as a great starting point for this technique, since it has each of those components.
# Access the experiment results from the first round as a dataframeground_truth_df = initial_experiment.as_dataframe()# Sample 10 examples to use as meta prompting examplesground_truth_df = ground_truth_df[:10]# Create a new column with the examples in a single stringground_truth_df["example"] = ground_truth_df.apply( lambda row: f"Input: {row['input']}\nOutput: {row['output']}\nExpected Output: {row['expected']}", axis=1,)ground_truth_df.head()
Now construct a new prompt that will be used to generate a new prompt.
meta_prompt = """You are an expert prompt engineer. You are given a prompt, and a list of examples.Your job is to generate a new prompt that will improve the performance of the model.Here are the examples:{examples}Here is the original prompt:{prompt}Here is the new prompt:"""original_base_prompt = ( prompt.format(variables={"prompt": "example prompt"}).get("messages")[0].get("content"))client = OpenAI()response = client.chat.completions.create( model="gpt-3.5-turbo", messages=[ { "role": "user", "content": meta_prompt.format( prompt=original_base_prompt, examples=ground_truth_df["example"].to_string() ), } ],)new_prompt = response.choices[0].message.content.strip()
Prompt gradient optimization is a technique that uses the gradient of the prompt to optimize individual components of the prompt using embeddings. It involves:
Converting the prompt into an embedding.
Comparing the outputs of successful and failed prompts to find the gradient direction.
Moving in the gradient direction to optimize the prompt.
Here you’ll define a function to get embeddings for prompts, and then use that function to calculate the gradient direction between successful and failed prompts.
import numpy as np# First we'll define a function to get embeddings for promptsdef get_embedding(text): client = OpenAI() response = client.embeddings.create(model="text-embedding-ada-002", input=text) return response.data[0].embedding# Function to calculate gradient direction between successful and failed promptsdef calculate_prompt_gradient(successful_prompts, failed_prompts): # Get embeddings for successful and failed prompts successful_embeddings = [get_embedding(p) for p in successful_prompts] failed_embeddings = [get_embedding(p) for p in failed_prompts] # Calculate average embeddings avg_successful = np.mean(successful_embeddings, axis=0) avg_failed = np.mean(failed_embeddings, axis=0) # Calculate gradient direction gradient = avg_successful - avg_failed return gradient / np.linalg.norm(gradient)# Get successful and failed examples from our datasetsuccessful_examples = ( ground_truth_df[ground_truth_df["output"] == ground_truth_df["expected"].get("type")]["input"] .apply(lambda x: x["prompt"]) .tolist())failed_examples = ( ground_truth_df[ground_truth_df["output"] != ground_truth_df["expected"].get("type")]["input"] .apply(lambda x: x["prompt"]) .tolist())# Calculate the gradient directiongradient = calculate_prompt_gradient(successful_examples[:5], failed_examples[:5])# Function to optimize a prompt using the gradientdef optimize_prompt(base_prompt, gradient, step_size=0.1): # Get base embedding base_embedding = get_embedding(base_prompt) # Move in gradient direction optimized_embedding = base_embedding + step_size * gradient # Use GPT to convert the optimized embedding back to text client = OpenAI() response = client.chat.completions.create( model="gpt-3.5-turbo", messages=[ { "role": "system", "content": "You are helping to optimize prompts. Given the original prompt and its embedding, generate a new version that maintains the core meaning but moves in the direction of the optimized embedding.", }, { "role": "user", "content": f"Original prompt: {base_prompt}\nOptimized embedding direction: {optimized_embedding[:10]}...\nPlease generate an improved version that moves in this embedding direction.", }, ], ) return response.choices[0].message.content.strip()# Test the gradient-based optimizationgradient_prompt = optimize_prompt(original_base_prompt, gradient)
gradient_prompt
if r"\{examples\}" in gradient_prompt: gradient_prompt = gradient_prompt.format(examples=few_shot_examples)params = CompletionCreateParamsBase( model="gpt-3.5-turbo", temperature=0, messages=[ { "role": "system", "content": gradient_prompt, }, # if your meta prompt includes few shot examples, make sure to include them here {"role": "user", "content": "{{prompt}}"}, ],)gradient_prompt = PhoenixClient().prompts.create( name=prompt_identifier, prompt_description="Gradient prompt result", version=PromptVersion.from_openai(params),)
DSPy makes a series of calls to optimize the prompt. It can be useful to see these calls in action. To do this, you can instrument the DSPy library using the OpenInference SDK, which will send all calls to Phoenix. This is optional, but it can be useful to have.
from openinference.instrumentation.dspy import DSPyInstrumentorDSPyInstrumentor().instrument(tracer_provider=tracer_provider)
Now you’ll setup the DSPy language model and define a prompt classification task.
# Import DSPy and set up the language modelimport dspy# Configure DSPy to use OpenAIturbo = dspy.LM(model="gpt-3.5-turbo")dspy.settings.configure(lm=turbo)# Define the prompt classification taskclass PromptClassifier(dspy.Signature): """Classify if a prompt is benign or jailbreak.""" prompt = dspy.InputField() label = dspy.OutputField(desc="either 'benign' or 'jailbreak'")# Create the basic classifierclassifier = dspy.Predict(PromptClassifier)
Your classifier can now be used to make predictions as you would a normal LLM. It will expect a prompt input and will output a label prediction.
classifier(prompt=ds.iloc[0].prompt)
However, DSPy really shines when it comes to optimizing prompts. By defining a metric to measure successful runs, along with a training set of examples, you can use one of many different optimizers built into the library.In this case, you’ll use the MIPROv2 optimizer to find the best prompt for your task.
def validate_classification(example, prediction, trace=None): return example["label"] == prediction["label"]# Prepare training data from previous examplestrain_data = []for _, row in ground_truth_df.iterrows(): example = dspy.Example( prompt=row["input"]["prompt"], label=row["expected"]["type"] ).with_inputs("prompt") train_data.append(example)tp = dspy.MIPROv2(metric=validate_classification, auto="light")optimized_classifier = tp.compile(classifier, trainset=train_data)
DSPy takes care of our prompts in this case, however you could still save the resulting prompt value in Phoenix:
params = CompletionCreateParamsBase( model="gpt-3.5-turbo", temperature=0, messages=[ { "role": "system", "content": optimized_classifier.signature.instructions, }, # if your meta prompt includes few shot examples, make sure to include them here {"role": "user", "content": "{{prompt}}"}, ],)dspy_prompt = PhoenixClient().prompts.create( name=prompt_identifier, prompt_description="DSPy prompt result", version=PromptVersion.from_openai(params),)
Run experiment with DSPy-optimized classifier
Redefine the task, using the new prompt.
# Create evaluation function using optimized classifierdef test_dspy_prompt(input): result = optimized_classifier(prompt=input["prompt"]) return result.label
Prompt Optimization Technique #5: DSPy with GPT-4o
In the last example, you used GPT-3.5 Turbo to both run your pipeline, and optimize the prompt. However, you can also use a different model to optimize the prompt, and a different model to run your pipeline.It can be useful to use a more powerful model for your optimization step, and a cheaper or faster model for your pipeline.Here you’ll use GPT-4o to optimize the prompt, and keep GPT-3.5 Turbo as your pipeline model.
Run experiment with DSPy-optimized classifier using GPT-4o
Redefine the task, using the new prompt.
# Create evaluation function using optimized classifierdef test_dspy_prompt(input): result = optimized_classifier_using_gpt_4o(prompt=input["prompt"]) return result.label
# Run experiment with DSPy-optimized classifierdspy_experiment_using_gpt_4o = px_client.experiments.run_experiment( dataset=dataset, task=test_dspy_prompt, evaluators=[evaluate_response], experiment_description="Prompt Optimization Technique #5: DSPy Prompt Tuning with GPT-4o", experiment_name="dspy-optimization-gpt-4o", experiment_metadata={"prompt": "prompt_id=" + dspy_prompt.id},)
Results
And just like that, you’ve run a series of prompt optimization techniques to improve the performance of a jailbreak classification task, and compared the results using Phoenix.You should have a set of experiments that looks like this:
{"docsConfig":{"theme":"mint","$schema":"https://mintlify.com/docs.json","name":"Phoenix","colors":{"primary":"#00769E","light":"#39C7FF","dark":"#009DD2"},"logo":{"light":"https://mintcdn.com/arizeai-433a7140/fitXeWll7ezQrIcg/docs/logo/light.png?fit=max\u0026auto=format\u0026n=fitXeWll7ezQrIcg\u0026q=85\u0026s=78e92baf10dd9eced114496ccb88dbf1","dark":"https://mintcdn.com/arizeai-433a7140/fitXeWll7ezQrIcg/docs/logo/dark.png?fit=max\u0026auto=format\u0026n=fitXeWll7ezQrIcg\u0026q=85\u0026s=ef36a0515f1c575affb91f81496ce2f3"},"favicon":"https://mintcdn.com/arizeai-433a7140/fitXeWll7ezQrIcg/docs/favicon.png?fit=max\u0026auto=format\u0026n=fitXeWll7ezQrIcg\u0026q=85\u0026s=7c6d0d7347821750d998c57c334ca376","background":{"color":{"light":"#FFFFFF","dark":"#08090A"}},"navbar":{"links":[]},"footer":{"socials":{"slack":"https://join.slack.com/t/arize-ai/shared_invite/zt-3r07iavnk-ammtATWSlF0pSrd1DsMW7g","x":"https://x.com/ArizePhoenix","bluesky":"https://bsky.app/profile/arize-phoenix.bsky.social","linkedin":"https://www.linkedin.com/showcase/113218220","github":"https://github.com/Arize-ai/phoenix"},"links":[{"items":[{"label":"Blog","href":"https://arize.com/blog?cat=phoenix"}]},{"items":[{"label":"English","href":"/docs/phoenix"},{"label":"日本語 (Japanese)","href":"/docs/phoenix/documentation/jp"},{"label":"中文 (Chinese)","href":"/docs/phoenix/documentation/zh"}]}]},"seo":{"metatags":{"canonical":"https://arize.com"},"indexing":"all"},"fonts":{"heading":{"family":"Geist","weight":600},"body":{"family":"Geist"}},"styling":{"codeblocks":{"theme":{"light":"github-light-default","dark":"github-dark-default"}},"latex":true},"integrations":{"gtm":{"tagId":"GTM-W96XM35"}},"contextual":{"options":["copy","view","chatgpt","claude","perplexity"],"display":"header"},"interaction":{"drilldown":false},"navigation":{"global":{"anchors":[]},"pages":[]}},"deployment":{"deploymentMetadataValue":{"subdomain":"arizeai-433a7140","hasAudioNarration":false,"actualSubdomain":"arizeai-433a7140","customDomains":[],"gitSource":{"type":"github","owner":"arize-ai","repo":"phoenix","deployBranch":"main","contentDirectory":"","isPrivate":false,"mountPath":"","label":""},"feedback":{"thumbs":true,"edits":true,"issues":true},"entitlements":{"AI_CHAT":{"status":"ENABLED"},"REMOVE_BRANDING":{"status":"ENABLED"},"CONTEXTUAL_FEEDBACK":{"status":"ENABLED"},"CODE_SNIPPET_FEEDBACK":{"status":"ENABLED"},"WEB_MCP":{"status":"ENABLED"},"NARRATION":{"status":"ENABLED"}},"buildId":"6b007be1286e5a48326299e7","clientVersion":"0.0.4248","searchFilterCounts":[{"language":"default","tag":"Integrations","count":211},{"language":"default","tag":"Documentation","count":177},{"language":"default","tag":"Release Notes","count":176},{"language":"default","tag":"SDK \u0026 API Reference","count":145},{"language":"default","tag":"Rest API","count":106},{"language":"default","tag":"API Reference","count":105},{"language":"default","tag":"LLM Providers","count":71},{"language":"default","tag":"Python","count":69},{"language":"default","tag":"Tracing","count":64},{"language":"default","tag":"Evaluation","count":61}]},"assistantConfig":{"enableAskAiButton":true,"enableAskAiSearchBar":true,"deflection":{"enabled":true,"email":"support@arize.com"}},"basePath":"","hasChatPermissions":true,"hasWebMcpPermissions":true,"toggles":[{"name":"kill-isr","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-rbac-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-mintlify-slack","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-integrations-settings","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"rbac-expansion","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-pr-grouping","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-prefix","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"analytics-v7","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"user-flows-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"no-publish-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-onboarding","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving-live","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"stripTemperatureMiddleware","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"support-platform-pylon-zendesk","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"instant-rollback","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications-server","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-fable","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fixed-ai-action-pricing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"site-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automation-resolve-merge-conflict","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"scoped-publishing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"mintql","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"page-level-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-version-history-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"billing-page-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"full-page-search","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workspace-projects","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"assistant-v5","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fork-pr-previews","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"revalidation-jwt","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-sso-scim","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"hobby-abuse-scan","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"dashboard-inbox","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-signup","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-dual-write","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-enabled","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"approve-review-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-metadata","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"sso-setup-deadline-banner","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-serve","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"live-agents","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"starter-pricing","enabled":true,"variant":{"name":"control","enabled":true,"payload":{"type":"string","value":"control"},"feature_enabled":true},"impressionData":false},{"name":"llms-txt-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"wildcard-email-auth","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false}],"googleFontsInlined":true},"customCss":[{"filePath":"docs/style.css","content":"@import url('https://fonts.googleapis.com/css2?family=Geist+Mono:wght@300;400;500;600\u0026display=swap');html:root{--font-family-mono-custom: 'Geist Mono'}html:root{--gray-50: 247 247 248;--gray-100: 241 242 243;--gray-200: 228 229 232;--gray-300: 201 203 208;--gray-400: 161 164 171;--gray-500: 124 128 136;--gray-600: 86 90 98;--gray-700: 61 64 70;--gray-800: 43 45 49;--gray-900: 21 22 22;--gray-950: 8 9 10}html:root{--px-radius-sm: 2px;--px-radius-md: 6px;--px-radius-lg: 8px;--px-accent-bg: #009DD2}html:not(.dark){--px-bg: #FFFFFF;--px-surface: #F7F7F8;--px-code-bg: #F1F2F3;--px-chip: #E9EAEC;--px-line: rgba(8, 9, 10, 0.18);--px-t1: #08090A;--px-t2: #2B2D31;--px-t3: #3D4046;--px-accent: #00769E;--px-accent-line: rgba(0, 157, 210, 0.55);--px-accent-soft: #E7F3FD;--px-on-accent: #08090A;--px-note: #E8F3EC;--px-note-line: rgba(30, 70, 50, 0.24);--px-note-t: #1B4230;--px-warn: #FBEEE0;--px-warn-line: rgba(90, 50, 0, 0.24);--px-warn-t: #54300A;--px-blue: #E7F3FD;--px-blue-line: rgba(0, 82, 125, 0.24);--px-blue-t: #00527D;--px-danger: #FFEBE9;--px-danger-line: rgba(207, 34, 46, 0.28);--px-danger-t: #A40E26}html.dark{--px-bg: #08090A;--px-surface: #151616;--px-code-bg: #050506;--px-chip: #1E2226;--px-line: rgba(194, 196, 234, 0.20);--px-t1: #FFFFFF;--px-t2: #D5D7DB;--px-t3: #A1A4AB;--px-accent: #39C7FF;--px-accent-line: rgba(57, 199, 255, 0.45);--px-accent-soft: #122F3B;--px-on-accent: #08090A;--px-note: #10241A;--px-note-line: rgba(159, 224, 188, 0.30);--px-note-t: #9FE0BC;--px-warn: #2A1B0A;--px-warn-line: rgba(255, 201, 138, 0.30);--px-warn-t: #FFC98A;--px-blue: #0A1A24;--px-blue-line: rgba(57, 195, 255, 0.30);--px-blue-t: #97D8FF;--px-danger: #2A0F12;--px-danger-line: rgba(255, 138, 148, 0.30);--px-danger-t: #FFB3B8}#content{line-height:1.75}#page-title,#content h1{font-size:clamp(2rem, 1.4rem + 2.1vw, 2.875rem);line-height:1.12;letter-spacing:-0.02em;font-weight:600}#content h2,#content h3,#content h4{letter-spacing:-0.01em;font-weight:600}#content code,#content kbd,#content samp{font-feature-settings:'tnum' 1}#content a:not(.card){color:var(--px-accent);text-decoration-thickness:1px;text-underline-offset:3px}:focus-visible{outline:2px solid var(--px-accent-line);outline-offset:2px}html.dark :focus-visible{outline-color:var(--px-accent-line)}::selection{background:var(--px-accent-soft)}#navbar{background:var(--px-bg);border-bottom:1px solid var(--px-line)}#navbar div:has(>.nav-tabs){position:relative}#navbar div:has(>.nav-tabs)::before{content:'';position:absolute;top:0;left:50%;width:100vw;margin-left:-50vw;border-top:1px solid var(--px-line);pointer-events:none}.nav-tabs-item[data-active]{box-shadow:inset 0 -2px 0 var(--px-accent)}#search-bar-entry{background:var(--px-surface);border:1px solid var(--px-line);border-radius:var(--px-radius-md)}#assistant-entry{background:var(--px-surface);border:1px solid var(--px-accent-line);border-radius:var(--px-radius-md)}#assistant-entry svg{color:var(--px-accent)}#page-context-menu{font-size:0.9375rem;align-items:stretch}.page-context-dropdown{font-size:0.9375rem;align-items:stretch}.product-selector{position:relative;display:inline-flex;align-items:stretch;flex:0 0 auto}.product-selector--desktop{margin-left:0.75rem;align-self:center}#page-context-menu-button{padding:10px 14px;border-radius:var(--px-radius-md) 0 0 var(--px-radius-md);background:var(--px-surface);border-color:var(--px-line)}.page-context-dropdown__label{display:inline-flex;align-items:center;padding:10px 14px;border:1px solid var(--px-line);border-radius:var(--px-radius-md) 0 0 var(--px-radius-md);background:var(--px-surface);border-color:var(--px-line);color:var(--px-t1);font-family:inherit;font-weight:400;text-decoration:none}#page-context-menu-button>div{font-size:0.9375rem;gap:9px}#page-context-menu>button[aria-haspopup]{height:auto;align-self:stretch;aspect-ratio:auto;padding:10px 11px;border-radius:0 var(--px-radius-md) var(--px-radius-md) 0;background:var(--px-surface);border-color:var(--px-line)}.page-context-dropdown__chevron{display:inline-flex;align-items:center;height:auto;align-self:stretch;aspect-ratio:auto;padding:10px 11px;border:1px solid var(--px-line);border-left:0;border-radius:0 var(--px-radius-md) var(--px-radius-md) 0;background:var(--px-surface);color:var(--px-t3);cursor:pointer}.page-context-dropdown__chevron svg{display:block;width:1rem;height:1rem;transition:transform 150ms ease}.page-context-dropdown__chevron[aria-expanded=\"true\"] svg{transform:rotate(180deg)}.page-context-dropdown__menu{position:absolute;z-index:60;top:calc(100% + 0.55rem);left:0;min-width:12.5rem;padding:0.4rem;background:var(--px-bg);border:1px solid var(--px-line);border-radius:var(--px-radius-md);box-shadow:0 0.75rem 1.5rem rgb(8 9 10 / 0.18)}.page-context-dropdown__menu[hidden]{display:none}.page-context-dropdown__item{display:flex;align-items:center;justify-content:space-between;min-height:2.5rem;padding:0.55rem 0.65rem;color:var(--px-t1);border-radius:var(--px-radius-sm);text-decoration:none}.page-context-dropdown__item:hover,.page-context-dropdown__item:focus-visible{color:var(--px-accent);background:var(--px-accent-soft)}.page-context-dropdown__item[data-selected=\"true\"]{color:var(--px-accent);font-weight:500}.page-context-dropdown__item[data-selected=\"true\"]::after{content:'✓';margin-left:1.5rem}@media (max-width: 767px){#navbar .product-selector--desktop{display:none}#mobile-nav-content .product-selector--mobile{margin:0.5rem 0 1rem}#mobile-nav-content .page-context-dropdown__label{flex:1 1 auto}#mobile-nav-content .page-context-dropdown__menu{position:static;margin-top:0.4rem}}#sidebar-content li[data-active]>a{background:var(--px-accent-soft);color:var(--px-accent);border-radius:var(--px-radius-md);font-weight:500}.sidebar-group-header{font-weight:600;color:var(--px-t1)}ul.sidebar-group:has(>li[id=\"/docs/phoenix/get-started\"]),ul.sidebar-group:has(>li[id=\"/docs/phoenix/integrations/developer-tools/coding-agents\"]),ul.sidebar-group:has(>li[id=\"/docs/phoenix/pxi\"]){margin-top:1px}.toc-item[data-active],.toc-item[data-active] a{color:var(--px-accent);font-weight:400}.card.card{background:var(--px-surface);border-color:var(--px-line);border-radius:var(--px-radius-lg);box-shadow:none}.frame.frame{background:var(--px-surface);border:1px solid var(--px-line);border-radius:var(--px-radius-lg)}.accordion.accordion,.accordion-group.accordion-group{border-radius:var(--px-radius-md);border-color:var(--px-line)}.accordion.accordion{background:var(--px-surface)}.code-block.code-block,.code-group.code-group{background:var(--px-code-bg);border-color:var(--px-line);border-radius:var(--px-radius-md)}.code-block-background{background:var(--px-code-bg)!important}html.dark .code-block-background{background:var(--px-code-bg)!important}#content :not(pre)>code{background:var(--px-chip);border-radius:var(--px-radius-sm)}.code-block:has([data-floating-buttons]) [data-fade-overlay]{background-image:none}[data-floating-buttons] button{background:var(--px-chip)}.callout .code-block.code-block,.callout .code-group.code-group,.callout .code-block-background,html.dark .callout .code-block-background{background:var(--px-bg)!important}.callout{border-radius:var(--px-radius-lg)}.callout[data-callout-type=\"callout\"],.callout[data-callout-type=\"info\"]{background:var(--px-surface);border-color:var(--px-line)}.callout[data-callout-type=\"callout\"] [data-component-part=\"callout-content\"],.callout[data-callout-type=\"info\"] [data-component-part=\"callout-content\"]{color:var(--px-t2)!important}.callout[data-callout-type=\"tip\"],.callout[data-callout-type=\"check\"]{background:var(--px-note);border-color:var(--px-note-line)}.callout[data-callout-type=\"tip\"] [data-component-part=\"callout-content\"],.callout[data-callout-type=\"check\"] [data-component-part=\"callout-content\"]{color:var(--px-note-t)!important}.callout[data-callout-type=\"warning\"]{background:var(--px-warn);border-color:var(--px-warn-line)}.callout[data-callout-type=\"warning\"] [data-component-part=\"callout-content\"]{color:var(--px-warn-t)!important}.callout[data-callout-type=\"note\"]{background:var(--px-blue);border-color:var(--px-blue-line)}.callout[data-callout-type=\"note\"] [data-component-part=\"callout-content\"]{color:var(--px-blue-t)!important}.callout[data-callout-type=\"danger\"]{background:var(--px-danger);border-color:var(--px-danger-line)}.callout[data-callout-type=\"danger\"] [data-component-part=\"callout-content\"]{color:var(--px-danger-t)!important}[data-component-part=\"step-number\"]>div{background:var(--px-accent-soft);border:1px solid var(--px-accent-line);color:var(--px-accent);font-weight:500}.update.update-container{flex-direction:column;gap:0.5rem;border-top:1px solid var(--px-line)}.update.update-container:first-of-type{border-top:none}.update.update-container>div:first-child{width:100%;position:static}.update.update-container>div:last-child{width:100%;max-width:none}#content th{color:var(--px-t1);font-weight:600;border-bottom:1px solid var(--px-line)}#content td{border-bottom:1px solid var(--px-line)}footer{border-top:1px solid var(--px-line)}.sr-only{position:absolute;width:1px;height:1px;padding:0;margin:-1px;overflow:hidden;clip:rect(0, 0, 0, 0);white-space:nowrap;border:0}table{display:table!important}.step-title{font-size:1.5rem!important}.dark .invert-on-dark{filter:invert(1)}.release-update-read-more{display:inline-block;margin-top:0.75rem;font-weight:500;color:var(--primary);text-decoration:none}"}]}
{"page":{"pagePath":"docs/phoenix/cookbook/prompt-engineering/prompt-optimization","slug":"/docs/phoenix/cookbook/prompt-engineering/prompt-optimization","pageMetadata":{"title":"Prompt Optimization Techniques","description":"This tutorial will use Phoenix to compare the performance of different prompt optimization techniques.","href":"/docs/phoenix/cookbook/prompt-engineering/prompt-optimization"},"description":{"compiledSource":"\"use strict\";\nconst {jsx: _jsx} = arguments[0];\nconst {useMDXComponents: _provideComponents} = arguments[0];\nfunction _createMdxContent(props) {\n const _components = {\n p: \"p\",\n ..._provideComponents(),\n ...props.components\n };\n return _jsx(_components.p, {\n children: \"This tutorial will use Phoenix to compare the performance of different prompt optimization techniques.\"\n });\n}\nfunction MDXContent(props = {}) {\n const {wrapper: MDXLayout} = {\n ..._provideComponents(),\n ...props.components\n };\n return MDXLayout ? _jsx(MDXLayout, {\n ...props,\n children: _jsx(_createMdxContent, {\n ...props\n })\n }) : _createMdxContent(props);\n}\nreturn {\n default: MDXContent\n};\n","frontmatter":{},"scope":{}},"pageType":"default","mdx":{"mdxSource":{"compiledSource":"\"use strict\";const{Fragment:_Fragment,jsx:_jsx,jsxs:_jsxs}=arguments[0];const{useMDXComponents:_provideComponents}=arguments[0];function _createMdxContent(props){const _components={a:`a`,code:`code`,li:`li`,ol:`ol`,p:`p`,pre:`pre`,span:`span`,strong:`strong`,ul:`ul`,..._provideComponents(),...props.components},{Card,CodeBlock,Frame,Heading,OptimizedFrame,OptimizedImage,Warning}=_components;if(!Card)_missingMdxReference(`Card`,true);if(!CodeBlock)_missingMdxReference(`CodeBlock`,true);if(!Frame)_missingMdxReference(`Frame`,true);if(!Heading)_missingMdxReference(`Heading`,true);if(!OptimizedFrame)_missingMdxReference(`OptimizedFrame`,true);if(!OptimizedImage)_missingMdxReference(`OptimizedImage`,true);if(!Warning)_missingMdxReference(`Warning`,true);return _jsxs(_Fragment,{children:[_jsx(Frame,{children:_jsx(OptimizedFrame,{src:`https://cdn.iframe.ly/XKrGMjE`,width:1e3,height:400,allowFullScreen:true,scrolling:`no`,allow:`accelerometer *; clipboard-write *; encrypted-media *; gyroscope *; picture-in-picture *; web-share *;`})}),`\n`,_jsx(Card,{title:`Google Colab`,horizontal:true,href:`https://colab.research.google.com/github/Arize-ai/phoenix/blob/dbb46f76ef3292066858280d9c7718b674c6cebc/tutorials/prompts/prompt-optimization.ipynb`,icon:`https://storage.googleapis.com/arize-phoenix-assets/assets/images/phoenix-docs-images/gc.ico`,horizontal:true,children:_jsx(_components.p,{children:`colab.research.google.com`})}),`\n`,_jsx(_components.p,{children:`You’ll start by creating an experiment in Phoenix that can house the results of each of your resulting prompts. Next you’ll use a series of prompt optimization techniques to improve the performance of a jailbreak classification task. Each technique will be applied to the same base prompt, and the results will be compared using Phoenix.`}),`\n`,_jsx(_components.p,{children:`The techniques you’ll use are:`}),`\n`,_jsxs(_components.ul,{children:[`\n`,_jsxs(_components.li,{children:[`\n`,_jsxs(_components.p,{children:[_jsx(_components.strong,{children:`Few Shot Examples`}),`: Adding a few examples to the prompt to help the model understand the task.`]}),`\n`]}),`\n`,_jsxs(_components.li,{children:[`\n`,_jsxs(_components.p,{children:[_jsx(_components.strong,{children:`Meta Prompting`}),`: Prompting a model to generate a better prompt based on previous inputs, outputs, and expected outputs.`]}),`\n`]}),`\n`,_jsxs(_components.li,{children:[`\n`,_jsxs(_components.p,{children:[_jsx(_components.strong,{children:`Prompt Gradients`}),`: Using the gradient of the prompt to optimize individual components of the prompt using embeddings.`]}),`\n`]}),`\n`,_jsxs(_components.li,{children:[`\n`,_jsxs(_components.p,{children:[_jsx(_components.strong,{children:`DSPy Prompt Tuning`}),`: Using DSPy, an automated prompt tuning library, to optimize the prompt.`]}),`\n`]}),`\n`]}),`\n`,_jsx(Warning,{children:_jsx(_components.p,{children:`This tutorial requires and OpenAI API key.`})}),`\n`,_jsx(_components.p,{children:`Let’s get started!`}),`\n`,_jsx(Heading,{level:`4`,id:`setup-dependencies-\u0026-keys`,children:`Setup Dependencies \u0026 Keys`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`1`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`1`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`!`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`pip`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` install`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:` -q`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"arize-phoenix>=8.0.0\"`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` datasets`})]}),`\n`]})})}),`\n`,_jsxs(_components.p,{children:[`Point your code at the Phoenix instance you started. The endpoint below is the default for a local `,_jsx(_components.code,{children:`phoenix serve`}),`; for a deployment running elsewhere, use its hostname instead.`]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`6`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`6`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` os`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`os.environ[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"PHOENIX_COLLECTOR_ENDPOINT\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"http://localhost:6006\"`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# Only if the deployment has authentication enabled`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# os.environ[\"PHOENIX_API_KEY\"] = \"your-api-key\"`})}),`\n`]})})}),`\n`,_jsx(_components.p,{children:`Then provide your OpenAI key, which this tutorial uses for the LLM calls:`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`5`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`5`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` os`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`from`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` getpass `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` getpass`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`if`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` not`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` os.environ.get(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"OPENAI_API_KEY\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`):`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` os.environ[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"OPENAI_API_KEY\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` getpass(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"Enter your OpenAI API key: \"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})]}),`\n`]})})}),`\n`,_jsx(Heading,{level:`4`,id:`load-dataset-into-phoenix`,children:`Load Dataset into Phoenix`}),`\n`,_jsx(_components.p,{children:`Since we’ll be running a series of experiments, we’ll need a dataset of test cases that we can run each time. This dataset will be used to test the performance of each prompt optimization technique.`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`5`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`5`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`from`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` datasets `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` load_dataset`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`ds `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` load_dataset(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"jackhhao/jailbreak-classification\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"train\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`ds `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ds.to_pandas().sample(`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`50`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`ds.head()`})}),`\n`]})})}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`13`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`13`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` uuid`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`from`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` phoenix.client `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` Client`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`unique_id `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` uuid.uuid4()`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# Upload the dataset to Phoenix`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`px_client `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` Client()`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`dataset `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` px_client.datasets.create_dataset(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` dataframe`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`ds,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` input_keys`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` output_keys`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` name`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"jailbreak-classification-`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`unique_id`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`]})})}),`\n`,_jsx(_components.p,{children:`Next, you can define a base template for the prompt. We’ll also save this template to Phoenix, so it can be tracked, versioned, and reused across experiments.`}),`\n`,_jsx(CodeBlock,{filename:``,expandable:`true`,numberOfLines:`24`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`24`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`from`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` openai `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` OpenAI`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`from`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` openai.types.chat.completion_create_params `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` CompletionCreateParamsBase`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`from`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` phoenix.client.types `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` PromptVersion`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`params `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` CompletionCreateParamsBase(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` model`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"gpt-3.5-turbo\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` temperature`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` messages`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`[`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` {`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"role\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"system\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"content\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"You are an evaluator that decides whether a given prompt is a jailbreak risk. Only output 'benign' or 'jailbreak', no other words.\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` },`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` {`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"role\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"user\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"content\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`prompt`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`}}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`},`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ],`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`prompt_identifier `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"jailbreak-classification\"`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`prompt `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` px_client.prompts.create(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` name`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`prompt_identifier,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` prompt_description`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"A prompt for classifying whether a given prompt is a jailbreak risk.\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` version`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`PromptVersion.from_openai(params),`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`]})})}),`\n`,_jsx(_components.p,{children:`You should now see that prompt in Phoenix:`}),`\n`,_jsx(Frame,{caption:``,children:_jsx(OptimizedImage,{src:`https://storage.googleapis.com/arize-phoenix-assets/assets/images/phoenix-docs-images/b63d2eb3-image.jpeg`,alt:`Initial evaluation prompt template and model configuration in Phoenix`})}),`\n`,_jsx(_components.p,{children:`Next you’ll need a task and evaluator for the experiment. A task is a function that will be run across each example in the dataset. The task is also the piece of your code that you’ll change between each run of the experiment. To start off, the task is simply a call to GPT 3.5 Turbo with a basic prompt.`}),`\n`,_jsx(_components.p,{children:`You’ll also need an evaluator that will be used to test the performance of the task. The evaluator will be run across each example in the dataset after the task has been run. Here, because you have ground truth labels, you can use a simple function to check if the output of the task matches the expected output.`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`8`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`8`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`def`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#D2A8FF`},children:` test_prompt`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(input):`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` client `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` OpenAI()`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` resp `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` client.chat.completions.create(`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`**`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`prompt.format(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`variables`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`input`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]}))`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` return`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` resp.choices[`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`].message.content.strip()`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`def`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#D2A8FF`},children:` evaluate_response`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(output, expected):`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` return`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` output.lower() `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`==`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` expected[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`].lower()`})]}),`\n`]})})}),`\n`,_jsx(_components.p,{children:`You can also instrument your code to send all models calls to Phoenix. This isn’t necessary for the experiment to run, but it does mean all your experiment task runs will be tracked in Phoenix. The overall experiment score and evaluator runs will be tracked regardless of whether you instrument your code or not.`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`6`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`6`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`from`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` openinference.instrumentation.openai `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` OpenAIInstrumentor`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`from`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` phoenix.otel `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` register`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`tracer_provider `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` register(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`project_name`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt-optimization\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`OpenAIInstrumentor().instrument(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`tracer_provider`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`tracer_provider)`})]}),`\n`]})})}),`\n`,_jsx(_components.p,{children:`Now you can run the initial experiment. This will be the base prompt that you’ll be optimizing.`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`12`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`12`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` nest_asyncio`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`nest_asyncio.apply()`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`initial_experiment `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` px_client.experiments.run_experiment(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` dataset`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`dataset,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` task`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`test_prompt,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` evaluators`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`[evaluate_response],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` experiment_description`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"Initial base prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` experiment_name`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"initial-prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` experiment_metadata`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt_id=\"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` +`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` prompt.id},`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`]})})}),`\n`,_jsx(_components.p,{children:`You should now see the initial experiment results in Phoenix:`}),`\n`,_jsx(Frame,{caption:`1st experiment results`,children:_jsx(OptimizedImage,{src:`https://storage.googleapis.com/arize-phoenix-assets/assets/images/phoenix-docs-images/0e581df6-image.jpeg`,alt:`Phoenix experiment results for the initial evaluation prompt`})}),`\n`,_jsx(Heading,{level:`2`,id:`prompt-optimization-technique-#1-few-shot-examples`,children:`Prompt Optimization Technique #1: Few Shot Examples`}),`\n`,_jsx(_components.p,{children:`One common prompt optimization technique is to use few shot examples to guide the model’s behavior.`}),`\n`,_jsx(_components.p,{children:`Here you can add few shot examples to the prompt to help improve performance. Conveniently, the dataset you uploaded in the last step contains a test set that you can use for this purpose.`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`6`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`6`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`from`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` datasets `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` load_dataset`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`ds_test `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` load_dataset(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"jackhhao/jailbreak-classification\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)[`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"test\"`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`] `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# this time, load in the test set instead of the training set`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`few_shot_examples `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ds_test.to_pandas().sample(`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`10`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})]}),`\n`]})})}),`\n`,_jsx(_components.p,{children:`Define a new prompt that includes the few shot examples. Prompts in Phoenix are automatically versioned, so saving the prompt with the same name will create a new version that can be used.`}),`\n`,_jsx(CodeBlock,{filename:``,expandable:`true`,numberOfLines:`22`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`22`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`few_shot_template `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"\"\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`You are an evaluator that decides whether a given prompt is a jailbreak risk. Only output \"benign\" or \"jailbreak\", no other words.`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`Here are some examples of prompts and responses:`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{examples}`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"\"\"`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`params `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` CompletionCreateParamsBase(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` model`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"gpt-3.5-turbo\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` temperature`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` messages`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`[`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` {`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"role\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"system\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"content\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: few_shot_template.format(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`examples`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`few_shot_examples)},`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` {`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"role\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"user\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"content\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`prompt`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`}}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`},`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ],`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`few_shot_prompt `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` PhoenixClient().prompts.create(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` name`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`prompt_identifier,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` prompt_description`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"Few shot prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` version`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`PromptVersion.from_openai(params),`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`]})})}),`\n`,_jsx(_components.p,{children:`You’ll notice you now have a new version of the prompt in Phoenix:`}),`\n`,_jsx(Frame,{caption:``,children:_jsx(OptimizedImage,{src:`https://storage.googleapis.com/arize-phoenix-assets/assets/images/phoenix-docs-images/4889cd2f-image.jpeg`,alt:`Saved evaluation prompt template with few-shot examples in Phoenix`})}),`\n`,_jsx(_components.p,{children:`Define a new task with your new prompt:`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`5`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`5`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`def`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#D2A8FF`},children:` test_prompt`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(input):`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` client `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` OpenAI()`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` prompt_vars `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` {`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`input`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]}`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` resp `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` client.chat.completions.create(`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`**`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`few_shot_prompt.format(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`variables`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`prompt_vars))`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` return`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` resp.choices[`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`].message.content.strip()`})]}),`\n`]})})}),`\n`,_jsx(_components.p,{children:`Now you can run another experiment with the new prompt. The dataset of test cases and the evaluator will be the same as the previous experiment.`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`8`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`8`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`few_shot_experiment `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` px_client.experiments.run_experiment(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` dataset`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`dataset,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` task`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`test_prompt,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` evaluators`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`[evaluate_response],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` experiment_description`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"Prompt Optimization Technique #1: Few Shot Examples\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` experiment_name`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"few-shot-examples\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` experiment_metadata`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt_id=\"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` +`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` few_shot_prompt.id},`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`]})})}),`\n`,_jsx(Heading,{level:`2`,id:`prompt-optimization-technique-#2-meta-prompting`,children:`Prompt Optimization Technique #2: Meta Prompting`}),`\n`,_jsx(_components.p,{children:`Meta prompting involves prompting a model to generate a better prompt, based on previous inputs, outputs, and expected outputs.`}),`\n`,_jsx(_components.p,{children:`The experiment from round 1 serves as a great starting point for this technique, since it has each of those components.`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`12`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`12`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# Access the experiment results from the first round as a dataframe`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`ground_truth_df `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` initial_experiment.as_dataframe()`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# Sample 10 examples to use as meta prompting examples`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`ground_truth_df `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ground_truth_df[:`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`10`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# Create a new column with the examples in a single string`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`ground_truth_df[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"example\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ground_truth_df.apply(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` lambda`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` row: `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"Input: `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`row[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`'input'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`}\\\\n`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`Output: `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`row[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`'output'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`}\\\\n`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`Expected Output: `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`row[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`'expected'`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` axis`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`1`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`ground_truth_df.head()`})}),`\n`]})})}),`\n`,_jsx(_components.p,{children:`Now construct a new prompt that will be used to generate a new prompt.`}),`\n`,_jsx(CodeBlock,{filename:``,expandable:`true`,numberOfLines:`33`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`33`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`meta_prompt `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"\"\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`You are an expert prompt engineer. You are given a prompt, and a list of examples.`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`Your job is to generate a new prompt that will improve the performance of the model.`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`Here are the examples:`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{examples}`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`Here is the original prompt:`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{prompt}`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`Here is the new prompt:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"\"\"`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`original_base_prompt `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` (`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` prompt.format(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`variables`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"example prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`}).get(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"messages\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)[`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`].get(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"content\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`client `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` OpenAI()`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`response `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` client.chat.completions.create(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` model`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"gpt-3.5-turbo\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` messages`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`[`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` {`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"role\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"user\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"content\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: meta_prompt.format(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` prompt`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`original_base_prompt, `}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`examples`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`ground_truth_df[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"example\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`].to_string()`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ),`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` }`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ],`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`new_prompt `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` response.choices[`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`].message.content.strip()`})]}),`\n`]})})}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`1`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`1`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`new_prompt`})}),`\n`]})})}),`\n`,_jsx(_components.p,{children:`Now save that as a prompt in Phoenix:`}),`\n`,_jsx(CodeBlock,{filename:``,expandable:`true`,numberOfLines:`17`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`17`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`if`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` r`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-light-font-weight\":`bold`,\"--shiki-dark\":`#7EE787`,\"--shiki-dark-font-weight\":`bold`},children:`\\\\{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`examples`}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-light-font-weight\":`bold`,\"--shiki-dark\":`#7EE787`,\"--shiki-dark-font-weight\":`bold`},children:`\\\\}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` in`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` new_prompt:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` new_prompt `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` new_prompt.format(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`examples`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`few_shot_examples)`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`params `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` CompletionCreateParamsBase(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` model`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"gpt-3.5-turbo\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` temperature`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` messages`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`[`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` {`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"role\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"system\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"content\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: new_prompt},`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` {`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"role\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"user\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"content\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`prompt`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`}}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`},`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ],`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`meta_prompt_result `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` PhoenixClient().prompts.create(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` name`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`prompt_identifier,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` prompt_description`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"Meta prompt result\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` version`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`PromptVersion.from_openai(params),`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`]})})}),`\n`,_jsx(Heading,{level:`4`,id:`run-this-new-prompt-through-the-same-experiment`,children:`Run this new prompt through the same experiment`}),`\n`,_jsx(_components.p,{children:`Redefine the task, using the new prompt.`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`6`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`6`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`def`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#D2A8FF`},children:` test_prompt`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(input):`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` client `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` OpenAI()`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` resp `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` client.chat.completions.create(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` **`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`meta_prompt_result.format(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`variables`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`input`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]})`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` )`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` return`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` resp.choices[`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`].message.content.strip()`})]}),`\n`]})})}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`8`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`8`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`meta_prompting_experiment `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` px_client.experiments.run_experiment(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` dataset`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`dataset,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` task`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`test_prompt,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` evaluators`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`[evaluate_response],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` experiment_description`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"Prompt Optimization Technique #2: Meta Prompting\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` experiment_name`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"meta-prompting\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` experiment_metadata`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt_id=\"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` +`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` meta_prompt_result.id},`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`]})})}),`\n`,_jsx(Heading,{level:`2`,id:`prompt-optimization-technique-#3-prompt-gradient-optimization`,children:`Prompt Optimization Technique #3: Prompt Gradient Optimization`}),`\n`,_jsx(_components.p,{children:`Prompt gradient optimization is a technique that uses the gradient of the prompt to optimize individual components of the prompt using embeddings. It involves:`}),`\n`,_jsxs(_components.ol,{children:[`\n`,_jsxs(_components.li,{children:[`\n`,_jsx(_components.p,{children:`Converting the prompt into an embedding.`}),`\n`]}),`\n`,_jsxs(_components.li,{children:[`\n`,_jsx(_components.p,{children:`Comparing the outputs of successful and failed prompts to find the gradient direction.`}),`\n`]}),`\n`,_jsxs(_components.li,{children:[`\n`,_jsx(_components.p,{children:`Moving in the gradient direction to optimize the prompt.`}),`\n`]}),`\n`]}),`\n`,_jsx(_components.p,{children:`Here you’ll define a function to get embeddings for prompts, and then use that function to calculate the gradient direction between successful and failed prompts.`}),`\n`,_jsx(CodeBlock,{filename:``,expandable:`true`,numberOfLines:`69`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`69`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` numpy `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`as`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` np`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# First we'll define a function to get embeddings for prompts`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`def`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#D2A8FF`},children:` get_embedding`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(text):`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` client `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` OpenAI()`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` response `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` client.embeddings.create(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`model`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"text-embedding-ada-002\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`, `}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`input`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`text)`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` return`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` response.data[`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`].embedding`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# Function to calculate gradient direction between successful and failed prompts`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`def`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#D2A8FF`},children:` calculate_prompt_gradient`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(successful_prompts, failed_prompts):`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:` # Get embeddings for successful and failed prompts`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` successful_embeddings `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` [get_embedding(p) `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`for`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` p `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`in`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` successful_prompts]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` failed_embeddings `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` [get_embedding(p) `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`for`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` p `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`in`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` failed_prompts]`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:` # Calculate average embeddings`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` avg_successful `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` np.mean(successful_embeddings, `}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`axis`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` avg_failed `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` np.mean(failed_embeddings, `}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`axis`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:` # Calculate gradient direction`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` gradient `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` avg_successful `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`-`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` avg_failed`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` return`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` gradient `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`/`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` np.linalg.norm(gradient)`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# Get successful and failed examples from our dataset`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`successful_examples `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` (`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ground_truth_df[ground_truth_df[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"output\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`==`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ground_truth_df[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"expected\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`].get(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)][`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"input\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` .apply(`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`lambda`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` x: x[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`])`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` .tolist()`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`failed_examples `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` (`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ground_truth_df[ground_truth_df[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"output\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`!=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ground_truth_df[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"expected\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`].get(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)][`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"input\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` .apply(`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`lambda`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` x: x[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`])`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` .tolist()`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# Calculate the gradient direction`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`gradient `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` calculate_prompt_gradient(successful_examples[:`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`5`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`], failed_examples[:`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`5`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`])`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# Function to optimize a prompt using the gradient`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`def`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#D2A8FF`},children:` optimize_prompt`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(base_prompt, gradient, step_size`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`0.1`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`):`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:` # Get base embedding`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` base_embedding `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` get_embedding(base_prompt)`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:` # Move in gradient direction`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` optimized_embedding `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` base_embedding `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`+`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` step_size `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`*`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` gradient`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:` # Use GPT to convert the optimized embedding back to text`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` client `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` OpenAI()`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` response `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` client.chat.completions.create(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` model`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"gpt-3.5-turbo\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` messages`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`[`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` {`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"role\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"system\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"content\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"You are helping to optimize prompts. Given the original prompt and its embedding, generate a new version that maintains the core meaning but moves in the direction of the optimized embedding.\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` },`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` {`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"role\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"user\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"content\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`f`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"Original prompt: `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`base_prompt`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`}\\\\n`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`Optimized embedding direction: `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`optimized_embedding[:`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`10`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`...`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`\\\\n`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`Please generate an improved version that moves in this embedding direction.\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` },`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ],`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` )`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` return`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` response.choices[`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`].message.content.strip()`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# Test the gradient-based optimization`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`gradient_prompt `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` optimize_prompt(original_base_prompt, gradient)`})]}),`\n`]})})}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`1`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`1`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`gradient_prompt`})}),`\n`]})})}),`\n`,_jsx(CodeBlock,{filename:``,expandable:`true`,numberOfLines:`20`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`20`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`if`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` r`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-light-font-weight\":`bold`,\"--shiki-dark\":`#7EE787`,\"--shiki-dark-font-weight\":`bold`},children:`\\\\{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`examples`}),_jsx(_components.span,{style:{color:`#116329`,\"--shiki-light-font-weight\":`bold`,\"--shiki-dark\":`#7EE787`,\"--shiki-dark-font-weight\":`bold`},children:`\\\\}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` in`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` gradient_prompt:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` gradient_prompt `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` gradient_prompt.format(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`examples`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`few_shot_examples)`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`params `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` CompletionCreateParamsBase(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` model`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"gpt-3.5-turbo\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` temperature`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` messages`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`[`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` {`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"role\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"system\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"content\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: gradient_prompt,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` }, `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# if your meta prompt includes few shot examples, make sure to include them here`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` {`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"role\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"user\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"content\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`prompt`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`}}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`},`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ],`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`gradient_prompt `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` PhoenixClient().prompts.create(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` name`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`prompt_identifier,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` prompt_description`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"Gradient prompt result\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` version`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`PromptVersion.from_openai(params),`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`]})})}),`\n`,_jsx(Heading,{level:`4`,id:`run-experiment-with-gradient-optimized-prompt`,children:`Run experiment with gradient-optimized prompt`}),`\n`,_jsx(_components.p,{children:`Redefine the task, using the new prompt.`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`6`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`6`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`def`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#D2A8FF`},children:` test_gradient_prompt`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(input):`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` client `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` OpenAI()`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` resp `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` client.chat.completions.create(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` **`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`gradient_prompt.format(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`variables`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`input`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]})`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` )`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` return`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` resp.choices[`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`].message.content.strip()`})]}),`\n`]})})}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`8`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`8`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`gradient_experiment `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` px_client.experiments.run_experiment(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` dataset`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`dataset,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` task`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`test_gradient_prompt,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` evaluators`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`[evaluate_response],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` experiment_description`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"Prompt Optimization Technique #3: Prompt Gradients\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` experiment_name`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"gradient-optimization\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` experiment_metadata`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt_id=\"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` +`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` gradient_prompt.id},`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`]})})}),`\n`,_jsx(Heading,{level:`2`,id:`prompt-optimization-technique-#4-prompt-tuning-with-dspy`,children:`Prompt Optimization Technique #4: Prompt Tuning with DSPy`}),`\n`,_jsxs(_components.p,{children:[`Finally, you can use an optimization library to optimize the prompt, like DSPy. `,_jsx(_components.a,{href:`https://github.com/stanfordnlp/dspy`,children:`DSPy`}),` supports each of the techniques you’ve used so far, and more.`]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`1`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`1`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`!`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`pip`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` install`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:` -q`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` dspy`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` openinference-instrumentation-dspy`})]}),`\n`]})})}),`\n`,_jsx(_components.p,{children:`DSPy makes a series of calls to optimize the prompt. It can be useful to see these calls in action. To do this, you can instrument the DSPy library using the OpenInference SDK, which will send all calls to Phoenix. This is optional, but it can be useful to have.`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`3`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`3`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`from`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` openinference.instrumentation.dspy `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` DSPyInstrumentor`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`DSPyInstrumentor().instrument(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`tracer_provider`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`tracer_provider)`})]}),`\n`]})})}),`\n`,_jsx(_components.p,{children:`Now you’ll setup the DSPy language model and define a prompt classification task.`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`18`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`18`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# Import DSPy and set up the language model`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` dspy`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# Configure DSPy to use OpenAI`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`turbo `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` dspy.LM(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`model`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"gpt-3.5-turbo\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`dspy.settings.configure(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`lm`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`turbo)`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# Define the prompt classification task`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`class`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` PromptClassifier`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`dspy`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`.`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`Signature`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`):`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"\"\"Classify if a prompt is benign or jailbreak.\"\"\"`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` prompt `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` dspy.InputField()`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` label `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` dspy.OutputField(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`desc`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"either 'benign' or 'jailbreak'\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# Create the basic classifier`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`classifier `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` dspy.Predict(PromptClassifier)`})]}),`\n`]})})}),`\n`,_jsxs(_components.p,{children:[`Your classifier can now be used to make predictions as you would a normal LLM. It will expect a `,_jsx(_components.code,{children:`prompt`}),` input and will output a `,_jsx(_components.code,{children:`label`}),` prediction.`]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`1`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`1`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`classifier(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`prompt`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`ds.iloc[`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`].prompt)`})]}),`\n`]})})}),`\n`,_jsx(_components.p,{children:`However, DSPy really shines when it comes to optimizing prompts. By defining a metric to measure successful runs, along with a training set of examples, you can use one of many different optimizers built into the library.`}),`\n`,_jsxs(_components.p,{children:[`In this case, you’ll use the `,_jsx(_components.code,{children:`MIPROv2`}),` optimizer to find the best prompt for your task.`]}),`\n`,_jsx(CodeBlock,{filename:``,expandable:`true`,numberOfLines:`14`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`14`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`def`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#D2A8FF`},children:` validate_classification`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(example, prediction, trace`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`None`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`):`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` return`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` example[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"label\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`==`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` prediction[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"label\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# Prepare training data from previous examples`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`train_data `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` []`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`for`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` _, row `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`in`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ground_truth_df.iterrows():`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` example `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` dspy.Example(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` prompt`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`row[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"input\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`][`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`], `}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`label`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`row[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"expected\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`][`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"type\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ).with_inputs(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` train_data.append(example)`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`tp `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` dspy.MIPROv2(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`metric`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`validate_classification, `}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`auto`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"light\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`optimized_classifier `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` tp.compile(classifier, `}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`trainset`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`train_data)`})]}),`\n`]})})}),`\n`,_jsx(_components.p,{children:`DSPy takes care of our prompts in this case, however you could still save the resulting prompt value in Phoenix:`}),`\n`,_jsx(CodeBlock,{filename:``,expandable:`true`,numberOfLines:`17`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`17`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`params `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` CompletionCreateParamsBase(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` model`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"gpt-3.5-turbo\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` temperature`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` messages`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`[`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` {`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"role\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"system\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:` \"content\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: optimized_classifier.signature.instructions,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` }, `}),_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# if your meta prompt includes few shot examples, make sure to include them here`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` {`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"role\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"user\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"content\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`{{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`prompt`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`}}`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`},`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` ],`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`dspy_prompt `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` PhoenixClient().prompts.create(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` name`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`prompt_identifier,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` prompt_description`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"DSPy prompt result\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` version`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`PromptVersion.from_openai(params),`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`]})})}),`\n`,_jsx(Heading,{level:`4`,id:`run-experiment-with-dspy-optimized-classifier`,children:`Run experiment with DSPy-optimized classifier`}),`\n`,_jsx(_components.p,{children:`Redefine the task, using the new prompt.`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`4`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`4`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# Create evaluation function using optimized classifier`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`def`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#D2A8FF`},children:` test_dspy_prompt`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(input):`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` result `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` optimized_classifier(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`prompt`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`input`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`])`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` return`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` result.label`})]}),`\n`]})})}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`9`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`9`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# Run experiment with DSPy-optimized classifier`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`dspy_experiment `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` px_client.experiments.run_experiment(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` dataset`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`dataset,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` task`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`test_dspy_prompt,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` evaluators`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`[evaluate_response],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` experiment_description`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"Prompt Optimization Technique #4: DSPy Prompt Tuning\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` experiment_name`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"dspy-optimization\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` experiment_metadata`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt_id=\"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` +`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` dspy_prompt.id},`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`]})})}),`\n`,_jsx(Heading,{level:`2`,id:`prompt-optimization-technique-#5-dspy-with-gpt-4o`,children:`Prompt Optimization Technique #5: DSPy with GPT-4o`}),`\n`,_jsx(_components.p,{children:`In the last example, you used GPT-3.5 Turbo to both run your pipeline, and optimize the prompt. However, you can also use a different model to optimize the prompt, and a different model to run your pipeline.`}),`\n`,_jsx(_components.p,{children:`It can be useful to use a more powerful model for your optimization step, and a cheaper or faster model for your pipeline.`}),`\n`,_jsx(_components.p,{children:`Here you’ll use GPT-4o to optimize the prompt, and keep GPT-3.5 Turbo as your pipeline model.`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`5`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`5`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`prompt_gen_lm `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` dspy.LM(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"gpt-4o\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`tp `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` dspy.MIPROv2(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` metric`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`validate_classification, `}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`auto`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"light\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`, `}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`prompt_model`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`prompt_gen_lm, `}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`task_model`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`turbo`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`optimized_classifier_using_gpt_4o `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` tp.compile(classifier, `}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`trainset`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`train_data)`})]}),`\n`]})})}),`\n`,_jsx(Heading,{level:`4`,id:`run-experiment-with-dspy-optimized-classifier-using-gpt-4o`,children:`Run experiment with DSPy-optimized classifier using GPT-4o`}),`\n`,_jsx(_components.p,{children:`Redefine the task, using the new prompt.`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`4`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`4`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# Create evaluation function using optimized classifier`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`def`}),_jsx(_components.span,{style:{color:`#8250DF`,\"--shiki-dark\":`#D2A8FF`},children:` test_dspy_prompt`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`(input):`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` result `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` optimized_classifier_using_gpt_4o(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:`prompt`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#79C0FF`},children:`input`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`[`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`])`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` return`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` result.label`})]}),`\n`]})})}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`9`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default github-dark-default`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0d1117`,color:`#1f2328`,\"--shiki-dark\":`#e6edf3`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`9`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#6E7781`,\"--shiki-dark\":`#8B949E`},children:`# Run experiment with DSPy-optimized classifier`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`dspy_experiment_using_gpt_4o `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` px_client.experiments.run_experiment(`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` dataset`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`dataset,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` task`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`test_dspy_prompt,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` evaluators`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`[evaluate_response],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` experiment_description`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"Prompt Optimization Technique #5: DSPy Prompt Tuning with GPT-4o\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` experiment_name`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"dspy-optimization-gpt-4o\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#FFA657`},children:` experiment_metadata`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#A5D6FF`},children:`\"prompt_id=\"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#FF7B72`},children:` +`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:` dspy_prompt.id},`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#E6EDF3`},children:`)`})}),`\n`]})})}),`\n`,_jsx(Heading,{level:`2`,id:`results`,children:`Results`}),`\n`,_jsx(_components.p,{children:`And just like that, you’ve run a series of prompt optimization techniques to improve the performance of a jailbreak classification task, and compared the results using Phoenix.`}),`\n`,_jsx(_components.p,{children:`You should have a set of experiments that looks like this:`}),`\n`,_jsx(Frame,{caption:``,children:_jsx(OptimizedImage,{src:`https://storage.googleapis.com/arize-phoenix-assets/assets/images/phoenix-docs-images/a1f9a7be-image.jpeg`,alt:`Phoenix experiment comparison after prompt optimization`})}),`\n`,_jsxs(_components.p,{children:[`From here, you can check out more `,_jsx(_components.a,{href:`/docs/phoenix/cookbook`,children:`examples on Phoenix`}),`, and if you haven’t already, `,_jsx(_components.a,{href:`https://github.com/Arize-ai/phoenix`,children:`please give us a star on GitHub!`}),` ⭐️`]})]})}function MDXContent(props={}){const{wrapper:MDXLayout}={..._provideComponents(),...props.components};return MDXLayout?_jsx(MDXLayout,{...props,children:_jsx(_createMdxContent,{...props})}):_createMdxContent(props)}return{default:MDXContent};function _missingMdxReference(id,component){throw new Error(`Expected `+(component?`component`:`object`)+\" `\"+id+\"` to be defined: you likely forgot to import, pass, or provide it.\")}","frontmatter":{},"scope":{"codeStyling":{"theme":{"light":"github-light-default","dark":"github-dark-default"}},"config":{},"pageMetadata":{"title":"Prompt Optimization Techniques","description":"This tutorial will use Phoenix to compare the performance of different prompt optimization techniques.","href":"/docs/phoenix/cookbook/prompt-engineering/prompt-optimization"}}},"mdxSourceWithNoJs":null},"mdxExtracts":{"tableOfContents":[{"title":"Setup Dependencies \u0026 Keys","slug":"setup-dependencies-\u0026-keys","depth":4,"children":[]},{"title":"Load Dataset into Phoenix","slug":"load-dataset-into-phoenix","depth":4,"children":[]},{"title":"Prompt Optimization Technique #1: Few Shot Examples","slug":"prompt-optimization-technique-#1-few-shot-examples","depth":2,"children":[]},{"title":"Prompt Optimization Technique #2: Meta Prompting","slug":"prompt-optimization-technique-#2-meta-prompting","depth":2,"children":[{"title":"Run this new prompt through the same experiment","slug":"run-this-new-prompt-through-the-same-experiment","depth":4,"children":[]}]},{"title":"Prompt Optimization Technique #3: Prompt Gradient Optimization","slug":"prompt-optimization-technique-#3-prompt-gradient-optimization","depth":2,"children":[{"title":"Run experiment with gradient-optimized prompt","slug":"run-experiment-with-gradient-optimized-prompt","depth":4,"children":[]}]},{"title":"Prompt Optimization Technique #4: Prompt Tuning with DSPy","slug":"prompt-optimization-technique-#4-prompt-tuning-with-dspy","depth":2,"children":[{"title":"Run experiment with DSPy-optimized classifier","slug":"run-experiment-with-dspy-optimized-classifier","depth":4,"children":[]}]},{"title":"Prompt Optimization Technique #5: DSPy with GPT-4o","slug":"prompt-optimization-technique-#5-dspy-with-gpt-4o","depth":2,"children":[{"title":"Run experiment with DSPy-optimized classifier using GPT-4o","slug":"run-experiment-with-dspy-optimized-classifier-using-gpt-4o","depth":4,"children":[]}]},{"title":"Results","slug":"results","depth":2,"children":[]}],"codeExamples":{}},"jsLinks":["/_mintlify/custom/arizeai-433a7140/ad8f81ded4a76bea/docs/analytics.js","/_mintlify/custom/arizeai-433a7140/f5010f618af65dc2/docs/product-selector.js"],"subdomain":"arizeai-433a7140","siteName":"Phoenix","title":"Prompt Optimization Techniques - Phoenix","navMaps":{"firstHrefInVersion":{},"firstHrefInLanguage":{}},"lastModified":"2026-11-03T19:52:26.629Z","originalFileLocation":"docs/phoenix/cookbook/prompt-engineering/prompt-optimization.mdx"},"headHtml":"\u003cscript data-mint-head>(function(){var d=document.documentElement,p=null;if(!p){try{p=localStorage.getItem(\"isDarkMode\")}catch(e){}if(p==='true')p='dark';else if(p==='false')p='light';if(p!=='light'\u0026\u0026p!=='dark'\u0026\u0026p!=='system')p=\"system\"}var r=p==='system'?(matchMedia(\"(prefers-color-scheme: dark)\").matches?'dark':'light'):p;d.classList.remove(r==='dark'?'light':'dark');d.classList.add(r);d.setAttribute(\"data-theme-preference\",p)})();\u003c/script>\u003cmeta data-mint-head name=\"description\" content=\"This tutorial will use Phoenix to compare the performance of different prompt optimization techniques.\">\u003cmeta data-mint-head name=\"application-name\" content=\"Phoenix\">\u003cmeta data-mint-head name=\"generator\" content=\"Mintlify\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/mintlify-assets/_mintlify/favicons/arizeai-433a7140/QaORPmVscwA-cjPG/_generated/favicon/android-chrome-192x192.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"apple-touch-icon\" type=\"image/png\" sizes=\"180x180\" href=\"/mintlify-assets/_mintlify/favicons/arizeai-433a7140/QaORPmVscwA-cjPG/_generated/favicon/apple-touch-icon.png\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/mintlify-assets/_mintlify/favicons/arizeai-433a7140/QaORPmVscwA-cjPG/_generated/favicon/favicon-16x16.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/mintlify-assets/_mintlify/favicons/arizeai-433a7140/QaORPmVscwA-cjPG/_generated/favicon/favicon-32x32.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/mintlify-assets/_mintlify/favicons/arizeai-433a7140/QaORPmVscwA-cjPG/_generated/favicon/favicon.ico\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/mintlify-assets/_mintlify/favicons/arizeai-433a7140/QaORPmVscwA-cjPG/_generated/favicon-dark/android-chrome-192x192.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/mintlify-assets/_mintlify/favicons/arizeai-433a7140/QaORPmVscwA-cjPG/_generated/favicon-dark/favicon-16x16.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/mintlify-assets/_mintlify/favicons/arizeai-433a7140/QaORPmVscwA-cjPG/_generated/favicon-dark/favicon-32x32.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/mintlify-assets/_mintlify/favicons/arizeai-433a7140/QaORPmVscwA-cjPG/_generated/favicon-dark/favicon.ico\" media=\"(prefers-color-scheme: dark)\">\u003cmeta data-mint-head property=\"og:title\" content=\"Prompt Optimization Techniques - Phoenix\">\u003cmeta data-mint-head property=\"og:description\" content=\"This tutorial will use Phoenix to compare the performance of different prompt optimization techniques.\">\u003cmeta data-mint-head property=\"og:site_name\" content=\"Phoenix\">\u003cmeta data-mint-head property=\"og:url\" content=\"https://arize.com/docs/phoenix/cookbook/prompt-engineering/prompt-optimization\">\u003cmeta data-mint-head property=\"og:type\" content=\"website\">\u003cmeta data-mint-head property=\"og:image\" content=\"https://arizeai-433a7140.mintlify.site/_mintlify/api/og/v2?division=AI+Engineering+Workflows\u0026amp;title=Prompt+Optimization+Techniques\u0026amp;description=This+tutorial+will+use+Phoenix+to+compare+the+performance+of+different+prompt+optimization+techniques.\u0026amp;theme=150ae3cca034578e18963c56\u0026amp;signature=e962e5d52b01343fb60a4a3b2535ac2cb7db642d920ae71c1eef11f0086ca304\">\u003cmeta data-mint-head property=\"og:image:width\" content=\"1200\">\u003cmeta data-mint-head property=\"og:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"twitter:card\" content=\"summary_large_image\">\u003cmeta data-mint-head name=\"twitter:title\" content=\"Prompt Optimization Techniques - Phoenix\">\u003cmeta data-mint-head name=\"twitter:description\" content=\"This tutorial will use Phoenix to compare the performance of different prompt optimization techniques.\">\u003cmeta data-mint-head name=\"twitter:image\" content=\"https://arizeai-433a7140.mintlify.site/_mintlify/api/og/v2?division=AI+Engineering+Workflows\u0026amp;title=Prompt+Optimization+Techniques\u0026amp;description=This+tutorial+will+use+Phoenix+to+compare+the+performance+of+different+prompt+optimization+techniques.\u0026amp;theme=150ae3cca034578e18963c56\u0026amp;signature=e962e5d52b01343fb60a4a3b2535ac2cb7db642d920ae71c1eef11f0086ca304\">\u003cmeta data-mint-head name=\"twitter:image:width\" content=\"1200\">\u003cmeta data-mint-head name=\"twitter:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"msapplication-config\" content=\"/mintlify-assets/_mintlify/favicons/arizeai-433a7140/QaORPmVscwA-cjPG/_generated/favicon/browserconfig.xml\">\u003cmeta data-mint-head name=\"apple-mobile-web-app-title\" content=\"Phoenix\">\u003cmeta data-mint-head name=\"msapplication-TileColor\" content=\"#00769E\">\u003clink data-mint-head rel=\"canonical\" href=\"https://arize.com/docs/phoenix/cookbook/prompt-engineering/prompt-optimization\">\u003clink data-mint-head rel=\"alternate\" type=\"application/xml\" href=\"/sitemap.xml\">\u003clink data-mint-head rel=\"alternate\" type=\"text/markdown\" href=\"/docs/phoenix/cookbook/prompt-engineering/prompt-optimization.md\">\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https://arize.com/#organization\",\"name\":\"Phoenix\",\"url\":\"https://arize.com\",\"logo\":{\"@type\":\"ImageObject\",\"url\":\"https://mintcdn.com/arizeai-433a7140/fitXeWll7ezQrIcg/docs/logo/light.png?fit=max\u0026auto=format\u0026n=fitXeWll7ezQrIcg\u0026q=85\u0026s=78e92baf10dd9eced114496ccb88dbf1\"}},{\"@type\":\"WebSite\",\"@id\":\"https://arize.com#website\",\"name\":\"Phoenix\",\"url\":\"https://arize.com\",\"publisher\":{\"@id\":\"https://arize.com/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https://arize.com/docs/phoenix/cookbook/prompt-engineering/prompt-optimization#webpage\",\"url\":\"https://arize.com/docs/phoenix/cookbook/prompt-engineering/prompt-optimization\",\"name\":\"Prompt Optimization Techniques\",\"description\":\"This tutorial will use Phoenix to compare the performance of different prompt optimization techniques.\",\"dateModified\":\"2026-11-03T19:52:26.629Z\",\"isPartOf\":{\"@id\":\"https://arize.com#website\"},\"breadcrumb\":{\"@id\":\"https://arize.com/docs/phoenix/cookbook/prompt-engineering/prompt-optimization#breadcrumb\"}},{\"@type\":\"BreadcrumbList\",\"@id\":\"https://arize.com/docs/phoenix/cookbook/prompt-engineering/prompt-optimization#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"AI Engineering Workflows\",\"item\":\"https://arize.com/docs/phoenix/cookbook/ai-engineering-workflows/iterative-evaluation-and-experimentation-workflow-python\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Prompt Optimization Techniques\",\"item\":\"https://arize.com/docs/phoenix/cookbook/prompt-engineering/prompt-optimization\"}]},{\"@type\":[\"Article\",\"TechArticle\"],\"@id\":\"https://arize.com/docs/phoenix/cookbook/prompt-engineering/prompt-optimization#article\",\"headline\":\"Prompt Optimization Techniques\",\"name\":\"Prompt Optimization Techniques\",\"description\":\"This tutorial will use Phoenix to compare the performance of different prompt optimization techniques.\",\"url\":\"https://arize.com/docs/phoenix/cookbook/prompt-engineering/prompt-optimization\",\"mainEntityOfPage\":{\"@id\":\"https://arize.com/docs/phoenix/cookbook/prompt-engineering/prompt-optimization#webpage\"},\"dateModified\":\"2026-11-03T19:52:26.629Z\",\"publisher\":{\"@id\":\"https://arize.com/#organization\"},\"isPartOf\":{\"@id\":\"https://arize.com#website\"}}]}\u003c/script>","agentPointerHref":"/docs/phoenix/cookbook/prompt-engineering/prompt-optimization.md","siteName":"Phoenix","nav":{"global":{"anchors":[]},"tabs":[{"tab":"Documentation","pages":[{"title":"Documentation","href":"/docs/phoenix"}]},{"tab":"Learn","pages":[{"title":"Learn","href":"/docs/phoenix/learn"}]},{"tab":"Integrations","pages":[{"title":"Integrations","href":"/docs/phoenix/integrations"}]},{"tab":"SDK \u0026 API Reference","pages":[{"title":"SDK \u0026 API Reference","href":"/docs/phoenix/sdk-api-reference"}]},{"tab":"Self-Hosting","pages":[{"title":"Self-Hosting","href":"/docs/phoenix/self-hosting"}]},{"tab":"Cookbooks","pages":[{"title":"Cookbooks","href":"/docs/phoenix/cookbook"},{"group":"Agent Workflow Patterns","pages":[{"title":"Agent Workflow Patterns","sidebarTitle":"Overview","href":"/docs/phoenix/cookbook/agent-workflow-patterns"},{"title":"AutoGen","href":"/docs/phoenix/cookbook/agent-workflow-patterns/autogen"},{"title":"CrewAI","href":"/docs/phoenix/cookbook/agent-workflow-patterns/crewai"},{"title":"Google GenAI SDK (manual orchestration)","href":"/docs/phoenix/cookbook/agent-workflow-patterns/google-genai-sdk-manual-orchestration"},{"title":"OpenAI Agents","href":"/docs/phoenix/cookbook/agent-workflow-patterns/openai-agents"},{"title":"LangGraph","href":"/docs/phoenix/cookbook/agent-workflow-patterns/langgraph"},{"title":"Smolagents","href":"/docs/phoenix/cookbook/agent-workflow-patterns/smolagents"}]},{"group":"AI Engineering Workflows","pages":[{"title":"Iterative Evaluation \u0026 Experimentation Workflow (Python)","href":"/docs/phoenix/cookbook/ai-engineering-workflows/iterative-evaluation-and-experimentation-workflow-python"},{"title":"Iterative Evaluation \u0026 Experimentation Workflow (TypeScript)","href":"/docs/phoenix/cookbook/ai-engineering-workflows/iterative-evaluation-and-experimentation-workflow-typescript"},{"title":"Aligning LLM Evals with Human Feedback (TypeScript)","description":"In this tutorial, we’ll run a Mastra agent and build a custom evaluator for it. The goal is to understand the workflow for creating evaluators that align with specific use cases.","href":"/docs/phoenix/cookbook/human-in-the-loop-workflows-annotations/aligning-llm-evals-with-human-annotations-typescript"},{"title":"Prompt Optimization Techniques","href":"/docs/phoenix/cookbook/prompt-engineering/prompt-optimization"},{"title":"Analyzing Customer Review Evals with Repetition Experiments","sidebarTitle":"Run Repetition Experiments","description":null,"href":"/docs/phoenix/cookbook/ai-engineering-workflows/analyzing-customer-review-evals-with-repetition-experiments"}]},{"group":"Tracing","pages":[{"title":"Identifying High-Signal Traces in Production","href":"/docs/phoenix/cookbook/tracing/identify-high-signal-traces"},{"title":"Agentic RAG Tracing","href":"/docs/phoenix/cookbook/tracing/agentic-rag-tracing"},{"title":"Trace and Evaluate Claude Managed Agents","href":"/docs/phoenix/cookbook/tracing/claude-managed-agents-guide"},{"title":"Generating Synthetic Datasets for LLM Evaluators \u0026 Agents","href":"/docs/phoenix/cookbook/tracing/generating-synthetic-datasets-for-llm-evaluators-and-agents"},{"title":"Structured Data Extraction","href":"/docs/phoenix/cookbook/tracing/structured-data-extraction"},{"title":"Product Recommendation Agent: Google Agent Engine \u0026 LangGraph","href":"/docs/phoenix/cookbook/tracing/product-recommendation-agent-google-agent-engine-and-langgraph"},{"title":"OpenInference Best Practices","href":"/docs/phoenix/cookbook/tracing/openinference-best-practices"},{"title":"More Cookbooks","href":"/docs/phoenix/cookbook/tracing/cookbooks"}]},{"group":"Human-in-the-Loop Workflows (Annotations)","pages":[{"title":"Using Human Annotations for Eval Driven Development","href":"/docs/phoenix/cookbook/human-in-the-loop-workflows-annotations/using-human-annotations-for-eval-driven-development"},{"title":"Aligning LLM Evals with Human Feedback (TypeScript)","href":"/docs/phoenix/cookbook/human-in-the-loop-workflows-annotations/aligning-llm-evals-with-human-annotations-typescript"}]},{"group":"Prompts","pages":[{"title":"Optimizing Coding Agent Prompts - Prompt Learning","href":"/docs/phoenix/cookbook/prompt-engineering/optimizing-coding-agent-prompts-prompt-learning"},{"title":"Optimizing Prompts for LLM Classification - Prompt Learning","href":"/docs/phoenix/cookbook/prompt-engineering/prompt-learning-optimizing-prompts-for-classification"},{"title":"Few Shot Prompting","href":"/docs/phoenix/cookbook/prompt-engineering/few-shot-prompting"},{"title":"ReAct Prompting","href":"/docs/phoenix/cookbook/prompt-engineering/react-prompting"},{"title":"Chain of Thought Prompting","description":null,"href":"/docs/phoenix/cookbook/prompt-engineering/chain-of-thought-prompting"},{"title":"Prompt Optimization Techniques","href":"/docs/phoenix/cookbook/prompt-engineering/prompt-optimization"},{"title":"LLM-as-a-Judge Prompt Optimization","description":null,"href":"/docs/phoenix/cookbook/prompt-engineering/llm-as-a-judge-prompt-optimization"}]},{"group":"Evaluation","pages":[{"title":"Creating a Custom LLM Evaluator with a Benchmark Dataset","sidebarTitle":"Write Your First Custom Eval","href":"/docs/phoenix/cookbook/evaluation/creating-a-custom-llm-evaluator-with-a-benchmark-dataset"},{"title":"OpenAI Agents SDK Cookbook","href":"/docs/phoenix/cookbook/evaluation/openai-agents-sdk-cookbook"},{"title":"Evaluate a Talk-to-Your-Data Agent","href":"/docs/phoenix/cookbook/evaluation/evaluate-an-agent"},{"title":"Trace-level Evaluation: Beyond Input/Output Checks","href":"/docs/phoenix/cookbook/evaluation/trace-level-evaluation"},{"title":"Session-level Evaluation: Scoring the Whole Conversation","href":"/docs/phoenix/cookbook/evaluation/session-level-evaluation"},{"title":"Evaluate RAG","href":"/docs/phoenix/cookbook/evaluation/evaluate-rag"},{"title":"Code Readability Evaluation","href":"/docs/phoenix/cookbook/evaluation/code-readability-evaluation"}]},{"group":"Guardrails \u0026 Safety","pages":[{"title":"Jailbreak and Prompt Injection Defense","href":"/docs/phoenix/cookbook/guardrails/jailbreak-and-prompt-injection-defense"},{"title":"Designing Realtime Guardrails: Input, Output, and the Cost of Blocking","href":"/docs/phoenix/cookbook/guardrails/designing-realtime-guardrails"}]},{"group":"Datasets \u0026 Experiments","pages":[{"title":"Analyzing Customer Review Evals with Repetition Experiments","href":"/docs/phoenix/cookbook/datasets-and-experiments/analyzing-customer-review-evals-with-repetition-experiments"},{"title":"Experiment with a Customer Support Agent","href":"/docs/phoenix/cookbook/datasets-and-experiments/experiment-with-a-customer-support-agent"},{"title":"Comparing LlamaIndex Query Engines with a Pairwise Evaluator","href":"/docs/phoenix/cookbook/datasets-and-experiments/comparing-llamaindex-query-engines-with-a-pairwise-evaluator"},{"title":"Prompt Template Iteration for a Summarization Service","href":"/docs/phoenix/cookbook/datasets-and-experiments/summarization"},{"title":"Text2SQL Experiments","href":"/docs/phoenix/cookbook/datasets-and-experiments/text2sql"},{"title":"Why Public Benchmarks Lie: Building Your Own Eval Harness","href":"/docs/phoenix/cookbook/datasets-and-experiments/building-your-own-eval-harness"},{"title":"More Cookbooks","href":"/docs/phoenix/cookbook/datasets-and-experiments/cookbooks"}]}]},{"tab":"Release Notes","pages":[{"title":"Release Notes","href":"/docs/phoenix/release-notes"}]}]},"basePath":"","title":"Prompt Optimization Techniques - Phoenix","deploymentId":"dpl_2pTxFtyBudBMYZiCw1MyvoA1wxpd","buildId":"dpl_2pTxFtyBudBMYZiCw1MyvoA1wxpd","dynamics":["07c022d89b","a74be98ba7"],"chunks":["Frame-B0oVBAXA.js","DynamicMarkdown-BvJIe00T.js","zoom-image-lightbox-BXthlMNf.js","lightbox-handle-r17bVUz3.js"]}