What Is Prompt Chaining?

Prompt chaining is a workflow where the output of one large language model (LLM) call becomes the input to the next call. In AI engineering, it lets you split a task into small steps you can test, inspect, and fix one at a time.

For example, suppose you need to summarize an invoice. Call 1 extracts the amount (4,200), due date (2026-11-01), and party (Acme) as JSON. Code checks those fields, Call 2 drafts “Acme owes $4,200, due 1 Nov 2026,” and Call 3 confirms that every number in the draft appears in the extracted fields.

Try Arize AX

Build better agents with Arize

Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.

Get started View AX docs

Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.

Why prompt chaining matters

A single large prompt that does everything at once is hard to test. When the output is wrong, you cannot tell which part of the task failed.

A chain gives each step its own prompt and a narrow job, such as extract, draft, check, or format. The developer fixes the sequence in advance, so the code decides what happens next, not the model.

Prompt chaining helps with:

• Testing prompts in isolation: Each prompt sees a small, defined input, so it is easier to evaluate.
• Catching errors early: Validate the output of one step before the next step uses it.
• Inspecting intermediate results: Log and review what each step produced.
• Retrying cheaply: Rerun one failed step without rerunning the whole chain.

Prompt chaining is one of the workflow patterns described in Anthropic’s Building Effective Agents. That guide recommends starting with simple, composable patterns like this before adding autonomy.

How prompt chaining works

A chain passes structured data from one call to the next. Each call receives only what it needs.

Consider a chain that summarizes a contract claim:

1. Extract: Step one pulls the claim fields out of the document as JSON.
2. Draft: Step two receives only those fields and writes a summary.
3. Verify: Step three receives the draft and the source fields, then flags any claim the fields do not support.

Because each prompt sees a small input, each prompt is shorter and easier to test.

Add code checks between calls

The handoff points in a chain are known in advance. That makes them a good place for ordinary code such as schema validation, allowlists, and arithmetic.

A code check is cheaper and more reliable than another model call for things code can verify. It is also much cheaper than debugging a finished output.

For more on making model outputs machine-checkable, see structured outputs.

What a minimal chain looks like

Here is the invoice chain from the opening as a flow:

document
  -> Call 1 (extract):  {"amount": 4200, "due_date": "2026-11-01", "party": "Acme"}
  -> deterministic check: amount is a positive number, due_date parses
  -> Call 2 (draft):    "Acme owes $4,200, due 1 Nov 2026."
  -> Call 3 (verify):   each noun/number in draft present in extracted fields?
  -> final answer, or route to human review if Call 3 fails

Notice the code check between Call 1 and Call 2. It stops a bad extraction before the drafting prompt ever sees it.

For more on how fixed workflows fit into larger systems, see agent workflow and agent orchestration.

How to implement a prompt chain

You can build a chain with plain functions. The following script runs the invoice chain with stand-in functions in place of real model calls, so it runs without an API.

It validates the extracted fields, then checks that every number in the draft comes from those fields. A deterministic check plays the role of Call 3 here. The last line tests a bad draft that says $2,400 instead of $4,200.

import json
import re
from datetime import date

def call_extract(document):
    # Stand-in for Call 1. A real chain would send an extraction prompt to a model.
    return '{"amount": 4200, "due_date": "2026-11-01", "party": "Acme"}'

def call_draft(fields):
    # Stand-in for Call 2. A real chain would send a drafting prompt to a model.
    return "Acme owes $4,200, due 1 Nov 2026."

def check_fields(raw):
    fields = json.loads(raw)
    assert isinstance(fields["amount"], (int, float)) and fields["amount"] > 0
    date.fromisoformat(fields["due_date"])
    return fields

def verify_draft(draft, fields):
    numbers = [n.replace(",", "") for n in re.findall(r"d[d,]*", draft)]
    due = date.fromisoformat(fields["due_date"])
    allowed = {str(fields["amount"]), str(due.day), str(due.year)}
    unsupported = [n for n in numbers if n not in allowed]
    return fields["party"] in draft and not unsupported, unsupported

fields = check_fields(call_extract("invoice text"))
print("check passed:", fields)

draft = call_draft(fields)
ok, unsupported = verify_draft(draft, fields)
print("draft:", draft)
print("verified:", ok, "| unsupported numbers:", unsupported)

bad_ok, bad_unsupported = verify_draft("Acme owes $2,400, due 1 Nov 2026.", fields)
print("bad draft verified:", bad_ok, "| unsupported numbers:", bad_unsupported)

Output:

check passed: {'amount': 4200, 'due_date': '2026-11-01', 'party': 'Acme'}
draft: Acme owes $4,200, due 1 Nov 2026.
verified: True | unsupported numbers: []
bad draft verified: False | unsupported numbers: ['2400']

The correct draft passes. The bad draft fails because 2,400 does not appear in the extracted fields, so the chain would route it to human review.

How to measure and debug a prompt chain

Instrument each model call as its own span. Record the prompt version, model, inputs, outputs, token counts, and latency. Then group all the spans under one trace for the whole run.

This follows the approach described in distributed tracing. It lets you answer three separate questions.

1. Step quality: On a fixed set of inputs, how often does this step produce an acceptable output? Use prompt evaluation on each step, not only on the final result.
2. Handoff quality: Is the upstream output in the shape the next prompt expects? Schema failures and silently cut-off fields show up here.
3. End-to-end quality: Does the final output complete the task? Track this with the same cases you use for LLM evaluation, and keep the per-step scores beside it.

When the final answer is wrong, replay the trace and find the first step whose output is wrong. Fix that step.

Changing the last prompt to make up for a bad extraction usually makes the system harder to reason about, not better.

For more on instrumenting chains, see the Phoenix documentation.

Common mistakes with prompt chaining

• Letting errors compound: If step one extracts the wrong amount, later steps usually polish that error instead of catching it. Only a step designed to check against the source will catch it.
• Ignoring serial latency: Three calls take roughly the sum of their times, plus any retries.
• Format drift at a handoff: A step returns prose where the next step expects JSON.
• Losing context between steps: A qualifier in the source document never reaches the drafting step.
• Debugging by final output only: Without per-step scores, you cannot see which stage regressed after a prompt change.
• Adding steps for appearance: Each step adds cost, latency, and another place for errors to enter. Keep chains as short as the task allows.

Prompt chaining vs. an agent loop

In a chain, the control flow is code, such as step 1, then step 2, then step 3. Any branches are ones you wrote.

In an agent loop, the model reads what happened and decides the next move. It might call a tool, try again, or stop, so the number and order of calls vary from run to run.

This difference matters for testing. A chain has a fixed set of paths to cover. An agent needs evaluation of its whole sequence of actions. For more on how agents decide their next step, see agent planning.

Choose a chain when you know the stages in advance. Choose an agent when the next action genuinely depends on what the model finds. Many production systems use a chain inside a single agent step.

Frequently asked questions

Is prompt chaining the same as a chain feature in an LLM library?

Not exactly. Libraries provide plumbing for passing outputs between calls. Prompt chaining is the design choice itself: a fixed sequence of model calls with defined handoffs. You can build it with plain functions.

How many steps should a prompt chain have?

A chain should have as few steps as its separate jobs require. Add a step when it has a distinct job you can evaluate on its own, such as extraction or verification. Do not add steps just to make the system look more structured.

Where should validation go in a prompt chain?

Validation goes at the handoffs. Check the schema and required fields after a step produces them and before the next prompt uses them. Add a source-checking step before any output a user will rely on.

When should I stop chaining and use an agent?

Switch to an agent when the right next action depends on intermediate results in ways you cannot list in advance. For example, an agent may need to choose which tool to query next based on what a previous query returned.

Putting prompt chaining to work

Start by writing down each stage of the task and what it must hand to the next stage. If a stage’s output can be checked with code, add that check at the handoff.

Build a small, fixed set of test inputs. Score each step on its own and score the final output, and keep both scores side by side.

Trace every run with one span per call. Tools such as Arize Phoenix can capture those spans, so when a final answer is wrong you can see which step introduced the error. For where fixed chains sit relative to more autonomous systems, see Arize’s overview of AI agents.