Few-shot prompting is a technique where you include a few input and output examples in a prompt so the model infers the task from them. In AI engineering, examples teach what instructions state badly, such as an exact output format, a house-specific judgment, or how to handle an edge case.
For example, suppose you classify support messages as billing, access, or other. The instruction alone does not say whether “I was charged twice after downgrading” counts as billing. Four short examples, including “Charged again after I canceled in May” labeled billing, show that a charge after a plan change is billing even when the word “invoice” never appears.
Build better agents with Arize
Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.
Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.
Why few-shot prompting matters
Instructions describe a task in words. Examples show it in the same input and output form the model will produce. For many tasks, showing is the faster way to get consistent results.
Few-shot prompting is useful for:
- Fixing output format: Show the exact fields, delimiters, or label set you expect.
- Teaching house rules: Demonstrate your label definitions, extraction quirks, or tone.
- Resolving edge cases: Settle an ambiguous case the instruction leaves open.
- Changing behavior quickly: Edit a prompt file instead of starting a training project.
Examples have a cost. They add tokens to every call, and they can bias the model toward the cases you demonstrated. Measure the trade-off against a prompt without examples.
How few-shot prompting works
A large language model (LLM) completes patterns. A prompt with examples sets up a pattern: these inputs map to outputs shaped like this. The model then continues that pattern for the new input.
This is called in-context learning. The behavior changes for this request only. Nothing is trained, and removing the examples removes the effect.
The vocabulary comes from a simple progression. A prompt with no examples is zero-shot, a prompt with one example is one-shot, and a prompt with a small set is few-shot. The Hugging Face prompting guide introduces this progression.
How to build a few-shot prompt
The following function builds the support-message prompt from the opening example. It keeps the instruction, the examples, and the new message in one consistent format.
INSTRUCTION = (
"Classify the message as billing, access, or other. "
"Reply with only the label."
)
EXAMPLES = [
("The invoice shows two seats we removed", "billing"),
("SSO login loops back to the start page", "access"),
("Charged again after I canceled in May", "billing"),
("Can you add dark mode?", "other"),
]
def build_prompt(message, examples=EXAMPLES):
lines = [INSTRUCTION, ""]
for text, label in examples:
lines.append(f'Message: "{text}"nLabel: {label}n')
lines.append(f'Message: "{message}"nLabel:')
return "n".join(lines)
print(build_prompt("I was charged twice after downgrading"))
Output:
Classify the message as billing, access, or other. Reply with only the label.
Message: "The invoice shows two seats we removed"
Label: billing
Message: "SSO login loops back to the start page"
Label: access
Message: "Charged again after I canceled in May"
Label: billing
Message: "Can you add dark mode?"
Label: other
Message: "I was charged twice after downgrading"
Label:
The examples add two rules the instruction never stated. A charge after a plan change counts as billing. A feature request is other, not access.
The examples also create a new boundary case. Consider a login problem caused by a billing hold. It now matches demonstrations from two different labels, so it is exactly the kind of case you should test.
Choose examples deliberately
These design decisions move results the most:
- Selection: Cover the decision boundaries your instructions handle badly, such as the ambiguous case, the refusal case, or the unusual but valid format. Near-duplicates of the easy case teach little.
- Correctness: The model imitates a wrong or inconsistent example. Review examples like test fixtures, including the reasoning style if you demonstrate reasoning.
- Ordering and balance: Models can overweight the most recent or the most common examples. If one class dominates the examples, expect it to dominate borderline outputs.
- Formatting: Keep delimiters and field structure identical across the examples and the real input. The pattern should be the task, not the punctuation.
Quality and coverage matter more than count. A handful of representative, correct examples usually beats a long, sloppy set.
Arize’s Phoenix cookbook recipe for few-shot prompting walks through building an example set in practice. The analysis of prompt optimization with few-shot examples covers refining one.
How to measure whether examples help
Build the evaluation set first, from real inputs. Keep it separate from your examples. If you test on your own examples, you measure whether the model memorized the prompt, not whether it can do the task.
Then run the zero-shot and few-shot versions on the full set. Use the same model and decoding settings for both. Compare the metric the task cares about, along with token cost and latency per call.
For a classifier, the metric is label accuracy. For extraction, it is field-level correctness. LLM evaluation covers judge and metric choices for outputs that cannot be matched exactly as strings.
Slice results by category
Few-shot gains often concentrate in the categories your examples cover. They can hide a loss somewhere else.
The following script compares accuracy per label for a small hypothetical result set. Each row holds the true label, the zero-shot prediction, and the few-shot prediction.
from collections import defaultdict
# Hypothetical results: (true label, zero-shot prediction, few-shot prediction)
results = [
("billing", "billing", "billing"),
("billing", "other", "billing"),
("billing", "access", "billing"),
("billing", "billing", "billing"),
("access", "access", "access"),
("access", "access", "billing"),
("access", "access", "access"),
("other", "access", "other"),
("other", "other", "other"),
]
def accuracy_by_label(rows, column):
hits, totals = defaultdict(int), defaultdict(int)
for row in rows:
label = row[0]
totals[label] += 1
hits[label] += row[column] == label
return {label: hits[label] / totals[label] for label in totals}
zero = accuracy_by_label(results, 1)
few = accuracy_by_label(results, 2)
for label in zero:
print(f"{label:<8} zero-shot {zero[label]:>4.0%} few-shot {few[label]:>4.0%}")
overall_zero = sum(r[1] == r[0] for r in results) / len(results)
overall_few = sum(r[2] == r[0] for r in results) / len(results)
print(f"overall zero-shot {overall_zero:>4.0%} few-shot {overall_few:>4.0%}")
Output:
billing zero-shot 50% few-shot 100%
access zero-shot 100% few-shot 67%
other zero-shot 50% few-shot 100%
overall zero-shot 67% few-shot 89%
The overall score improves from 67% to 89%. The access slice drops from 100% to 67%, because one access message is now labeled billing. Nine cases are far too few to draw a conclusion, but the per-slice view is what reveals the loss.
Inspect failures and trim the set
Inspect failing cases in traces. Check whether the model copied an example’s surface form.
For instance, suppose an access example is phrased like a password reset request. It might improve password reset messages while hurting other login errors. Try removing or replacing that example before you add more.
Keep the smallest example set that holds the gain. Every example is paid for on every call. It also competes for space with retrieved evidence and history, which you manage as part of your context.
Common mistakes with few-shot prompting
- Copying facts from examples: Models may copy a demonstrated answer’s details into an unrelated output.
- Over-applying a rule: A rule shown in one example can spread past its intended scope.
- Refusing unfamiliar cases: The model may refuse or stumble on inputs unlike any example.
- Crowding out evidence: Long example sets raise cost and latency and can push task evidence out of a fixed context budget.
- Leaking real data: If an example contains real customer data, it is now in every prompt and every trace. Use synthetic or redacted examples.
- Keeping a stale set: A set tuned for one model’s habits may steer its replacement differently. Revalidate the set when the model changes.
Few-shot prompting vs. zero-shot and fine-tuning
Zero-shot prompting gives instructions and no examples. It is cheaper per call and easier to maintain. With current models, it is often enough when the task is common and the output shape is simple.
Few-shot earns its tokens when the judgment is specific to your team, such as your label definitions, your extraction quirks, or your tone. Zero-shot is not a failed version of few-shot. It is the baseline every few-shot prompt has to beat.
Fine-tuning changes model weights on training data. It suits high-volume, stable tasks where the context cost of each call is high. It is also a training project, with its own data needs and regression testing.
Few-shot is the lighter intervention. It is editable in a prompt file and takes effect immediately. It is also easy to break with a careless edit, which is why example sets belong under prompt evaluation like any other prompt change.
Some prompt optimization methods search over instructions and example sets together. Judge that search on held-out cases, never on the examples themselves. For more, see prompt optimization.
Frequently asked questions
How many examples is “few-shot”?
There is no fixed count. One example is one-shot, and few-shot usually means a small set, often two to eight. Size the set to cover the distinct cases the model must tell apart, and add an example to fix a failing case, not to reach a number.
Should examples include reasoning, or just answers?
Match what you want the model to produce. If the output should be a bare label, long reasoning in the examples teaches the wrong shape. If the output benefits from visible steps, show concise steps in the form you want repeated.
Why did few-shot prompting make results worse?
The usual causes are an incorrect or ambiguous example, an imbalance between classes, or examples that do not represent real traffic. Another cause is an evaluation set that contains the examples themselves. Compare per-category results against the zero-shot baseline to find the damage.
When should I fine-tune instead of using few-shot prompting?
Consider fine-tuning when a stable, high-volume task still fails after a well-tested prompt. It also makes sense when example tokens cost more per call than training would, or when the behavior is too extensive to demonstrate. Until then, a versioned, evaluated example set is easier to inspect and roll back.
Can I test few-shot prompts on the examples I put in them?
No. Testing on your own examples measures memorization of the prompt, not task ability. Keep the evaluation set separate from the examples.
Putting few-shot prompting to work
Start with a zero-shot baseline on an evaluation set built from real inputs. Add a small number of correct examples that target the cases the baseline gets wrong.
Compare both versions on the same set, by category, along with token cost and latency. Keep only the examples that hold a measurable gain.
Version your example set with the prompt, and rerun the comparison when the model changes. Tools such as Arize Phoenix can trace each call, so you can see when the model copied an example instead of doing the task.
