What Is Code Mode?

Code mode is a pattern where an AI agent calls tools by writing and running code, instead of making one call at a time. It matters for agents because multi-step tasks can finish in fewer model turns, with less intermediate data passing through the model’s context.

For example, suppose an agent must find the ten active accounts with the most open incidents. With ordinary tool calling, it lists accounts, then requests usage for each one in a separate model turn, and every record passes through its context. In code mode, it writes one short program that loops over the accounts, filters to open incidents, and returns only the top ten rows.

Try Arize AX

Build better agents with Arize

Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.

Get started View AX docs

Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.

Why code mode matters for agents

Many agent tasks need several tool calls whose number and order depend on what earlier calls return. Code mode lets the agent express that logic once, as a program, and let a runtime carry it out.

Code mode is useful for:

• Filtering large results: Keep bulky intermediate records in the execution environment and return only what the model needs.
• Joining data across tools: Combine results from several application programming interfaces (APIs) without routing each record through the model.
• Handling pagination and loops: Iterate over pages or records in code instead of spending a model turn per call.
• Branching on intermediate data: Decide the next call based on what an earlier call returned.

For the underlying mechanism, see tool calling.

How code mode works

In a conventional tool-calling loop, the model emits one structured call. The application runs it and returns the result. The model then decides the next call.

That loop is clear and easy to constrain. But consider a task that needs twelve lookups, a join, and a filter. Every intermediate result passes through the model, and each step costs a model turn. The model’s context fills with records it only needed to pass along.

In code mode, the model writes a short program against the available tool or API functions. The program runs in an execution environment. It calls the tools, keeps intermediate values in variables, and returns a smaller result.

A search across accounts, a filter to active records, and a detail lookup for each match become one generated program. The model sees only the final answer.

Tool definitions still matter. The model needs accurate names, argument types, return shapes, and error behavior to write code that runs.

Where the term comes from

The term is widely credited to Cloudflare’s Code Mode post. Arize’s Code Mode article discusses the pattern. Anthropic’s code execution with MCP describes a related implementation.

Read each source for its own semantics. Terms differ between providers. Not every vendor executes, secures, or reports generated code in the same way.

For the surrounding architecture, see Model Context Protocol (MCP) and agent architecture.

Example: a generated program

The following function is the kind of code an agent might generate for the account task. It is illustrative. The accounts and usage_api objects stand in for real tool interfaces, so the block has no output to show.

# generated by the agent
def top_accounts_with_incidents(accounts, usage_api):
    matches = []
    for account in accounts.list(status="active"):
        usage = usage_api.summary(account_id=account["id"])
        if usage["open_incidents"] > 0:
            matches.append({"id": account["id"], "incidents": usage["open_incidents"]})
    return sorted(matches, key=lambda row: -row["incidents"])[:10]

The application records the code, each accounts.list and usage_api.summary call, and the ten rows returned. The model then explains or acts on those rows without receiving every account record.

How to inspect and debug code mode

Keep three records for every run:

• Generated code: The program the agent wrote.
• Tool calls: Each call the program made, with its arguments and its result or a result summary.
• Return value: The final value passed back to the model.

Each record catches different failures. If you keep only the final value, a wrong join or a silently empty filter is invisible. If you keep only the code, you cannot tell which call failed against live data.

Version the tool definitions available to the model beside the run. Code written against one return shape may fail after an interface change.

For more on tracing runs like this, see AI observability.

Test on known answers first

Test code mode on tasks with known answers before you use it on open-ended work. Include these cases:

• Empty results: A query that returns nothing.
• Paginated results: A result that spans several pages.
• Duplicate records: The same record returned more than once.
• Mid-run errors: A tool that returns an error halfway through.

Check that the generated code handles each case. A common bug is treating an empty first page as the full result.

Set limits on execution

Set time, memory, and call-count limits. Run generated code in a controlled environment with only the credentials the task needs.

These sandbox controls are a separate requirement. They are not a property of code mode itself. For more on isolated execution, see agent sandbox.

Common mistakes and limitations

Code mode inherits the usual risks of software, plus a few of its own.

• Ordinary code bugs: Generated code can use wrong field names, mishandle pagination, hide type errors, or filter out the records that matter.
• Runaway execution: Code can loop, call tools too often, or return a result so large that the context problem comes back. Limit call counts and result size, and require a stated return shape.
• Broad credentials: Running model-written code with production credentials and open network access gives a mistaken program wide reach. Limit credentials, destinations, and writable paths, and log attempted actions.
• Mistaking success for safety: A successful run does not show that the code was safe. It may simply not have tried anything harmful on that input.
• Disposable code: If you throw the code away, a later reviewer cannot reproduce a wrong answer. Keep the code and call trace with the run.
• Weak underlying tools: Code mode does not improve the tools. Slow, ambiguous, or unreliable APIs produce slow, unreliable programs.

Code mode vs. related patterns

Code mode vs. ordinary tool calling

The difference is where composition happens. In ordinary tool calling, the model builds the workflow one call per turn. In code mode, the model writes the whole composition once and the runtime executes it.

Ordinary tool calling is easier to approve step by step. Code mode handles data-dependent branching and bulk filtering with fewer model turns and less intermediate context.

Code mode vs. writing code for a human

Code mode is not the same as asking a model to write code for a person to run later. In code mode, execution is part of the agent’s own workflow, and the result feeds its next decision.

That makes error handling, limits, and logging part of the design from the start.

Code mode vs. vendor implementations

Related vendor patterns include programmatic tool calling and code execution with MCP. They share the same shape but may differ in available languages, tool exposure, timeout behavior, and how results return to the model.

Confirm those details in each provider’s documentation. Do not assume one provider’s behavior applies to another.

Code mode vs. fixed workflows

A workflow written by developers can be reviewed, tested, and versioned in advance. Generated code adapts to each request but must be checked at run time.

For stable, high-volume paths, a fixed workflow is often the better choice. Code mode fits best when the call pattern truly depends on what the data shows.

Frequently asked questions

When should an agent use code mode instead of single tool calls?

Use code mode when a task needs several calls whose number or order depends on intermediate data. It also helps when results need filtering or joining before the model sees them. For one or two predictable calls, the ordinary tool-calling loop is simpler to approve and debug.

Does code mode reduce cost?

It can, by cutting model turns and keeping bulky results out of the model’s context. It can also add cost through execution time, extra tool calls, and retries after code errors. Measure the whole task on a fixed set of cases, not only token count.

Is code mode the same as programmatic tool calling?

They are closely related, but naming and semantics vary by provider. Check how each provider exposes tools to code, where code runs, what limits apply, and how results return.

What should I log for a code mode run?

Log the generated code, the tool definition version, and each tool call with its timing and outcome. Also log resource use and the value returned to the model. That record lets you separate a code error from a tool error and reproduce the run.

Putting code mode to work

Start with one multi-step task that has a known answer, such as the account example above. Compare ordinary tool calling with code mode on model turns, cost, latency, and correctness.

Record the generated code, every tool call, and the return value for each run. Tracing tools such as Arize Phoenix can capture those calls as spans so you can see where a wrong answer came from. For how agents fit together more broadly, see the agent entry.