How to trace LiteLLM calls with Phoenix

Tracing LiteLLM calls with Phoenix takes a few lines of setup and captures every call automatically: the model requested, the complete input and output, token counts, latency, and any tool definitions or tool-call requests the model made. This tutorial walks through the setup for both the Python SDK and the Proxy Server, and shows what you actually see in the trace once it’s running.

LiteLLM routes calls across 100+ providers using a single OpenAI-compatible interface. When a call returns something unexpected, it’s hard to tell what prompt was sent, which model LiteLLM called, or whether a fallback ran. Phoenix tracing makes every call inspectable without changing how your application code works.

What does LiteLLM tracing capture automatically?

The OpenInference package openinference-instrumentation-litellm instruments the following LiteLLM SDK functions without any changes to your existing code. For the Proxy Server, see How to set up LiteLLM Proxy tracing.

• completion() and acompletion()
• completion_with_retries()
• responses() and aresponses() (new)
• Anthropic-style messages calls (new)
• embedding() and aembedding()
• image_generation() and aimage_generation()

Prerequisites

• Python 3.10+
• A Phoenix account (free signup) or a self-hosted Phoenix instance
• An API key for whichever provider you’re routing through LiteLLM

Step 1: Install the packages

pip install openinference-instrumentation-litellm arize-phoenix-otel litellm

Try Arize AX

Build better agents with Arize

Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.

Get started View AX docs

Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.

Step 2: Set your environment variables

If you run Phoenix yourself, set PHOENIX_COLLECTOR_ENDPOINT to your instance’s URL, for example http://localhost:6006.

export PHOENIX_API_KEY="your-phoenix-api-key"
export PHOENIX_COLLECTOR_ENDPOINT="your-phoenix-hostname"
export OPENAI_API_KEY="your-openai-api-key"

Step 3: Register the tracer and instrument LiteLLM

from phoenix.otel import register
from openinference.instrumentation.litellm import LiteLLMInstrumentor

tracer_provider = register(
    project_name="litellm-tracing",
)

LiteLLMInstrumentor().instrument(tracer_provider=tracer_provider)

Step 4: Make a call

import litellm

response = litellm.completion(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What causes a context window overflow?"}]
)

print(response)

What does a LiteLLM trace show in Phoenix?

Each LiteLLM call appears as a single span. On that span, you see the model passed into the call, the complete input and output, token usage, and latency. If the call included tools, the tool definitions you passed and the model’s tool-call request (which function it chose and the arguments it generated) are recorded as attributes on the same span. In the Phoenix trace view, you see:

• Model: The model argument passed into the call. With fallbacks=, the outer span keeps the model you requested. Each model LiteLLM tries appears as a child span, and failed attempts are marked as errors.
• The full input and output: The complete prompt and response
• Token usage: Input tokens and output tokens. The instrumentor also records cached, reasoning, and audio token counts when the provider returns them.
• Latency: Total time for the call
• Tool calls: The tool definitions sent with the request and the model’s tool-call request, both recorded as attributes on the call’s span.
• Cost: The cost LiteLLM calculates for the call, when it can price the model.

For agents, the span shows exactly which tool the model asked for and the arguments it generated. LiteLLM doesn’t run the tool, though, so a tool that fails or returns malformed data won’t appear as its own span. To see that, instrument the application code that runs the tool separately.

How to set up LiteLLM Proxy tracing

If your team runs LiteLLM as a centralized gateway serving multiple services or applications, instrument the proxy itself rather than each application. Traces flow from every downstream request automatically, with no per-service instrumentation required.

Set these environment variables before starting the proxy:

export PHOENIX_API_KEY="your-phoenix-api-key"
export PHOENIX_COLLECTOR_ENDPOINT="https://app.phoenix.arize.com/s/YOUR_SPACE/v1/traces"
export PHOENIX_PROJECT_NAME="litellm-proxy"

Replace YOUR_SPACE with your Space name, which you can find in your Phoenix Cloud URL. Unlike the SDK setup in Step 2, the proxy uses this URL exactly as given, so include the full path ending in /v1/traces. PHOENIX_PROJECT_NAME sets which Phoenix project receives the proxy traces. Without it, they go to the default project.

In your config.yaml:

litellm_settings:
  callbacks: ["arize_phoenix"]

Start the proxy as normal:

litellm --config /path/to/config.yaml

Every request routed through the proxy produces a trace, regardless of which application or team sent it. For platform teams that run LiteLLM as shared infrastructure, this gives you one place to instrument and one Phoenix project for all your services.

If you’re using Arize AX

If you use Arize AX, the same instrumentor works. Change the dependency and the register call:

pip install openinference-instrumentation-litellm arize-otel litellm

from arize.otel import register
from openinference.instrumentation.litellm import LiteLLMInstrumentor

tracer_provider = register(
    space_id="your-arize-space-id",
    api_key="your-arize-api-key",
    project_name="litellm-tracing",
)

LiteLLMInstrumentor().instrument(tracer_provider=tracer_provider)

Phoenix is free and open source: create a space and send your first LiteLLM trace in under five minutes.

FAQs: Tracing LiteLLM with Phoenix

Does LiteLLM Phoenix tracing work with multiple providers?

Yes. Each span records the model argument passed into that call, along with its latency and token usage. If a fallback fires and LiteLLM makes another instrumented call with the fallback model, that call appears as its own span with the fallback model’s name. A single span is never relabeled with whichever model ultimately served the request, so to see a fallback, look for the additional call in the trace.

What does a LiteLLM trace show in Phoenix?

Each call is captured as one span with the model passed into the call, the complete input prompt, the raw output, token counts, and latency. For calls that use tool calling, the tool definitions and the model’s tool-call request are recorded as attributes on that span.

Does LiteLLM tracing require changes to existing code?

No. The instrumentation wraps LiteLLM’s functions automatically after you call LiteLLMInstrumentor().instrument(). Your application code stays the same.

What is the difference between LiteLLM SDK tracing and Proxy tracing?

SDK instrumentation runs in-process and is the right choice when LiteLLM is embedded in a single application. Proxy instrumentation runs at the gateway level and traces every request that passes through it, regardless of which application sent it. For platform teams running LiteLLM as shared infrastructure, proxy instrumentation means one setup covers all downstream services.

Does LiteLLM Phoenix tracing support async calls?

Yes. acompletion() and aembedding() are both captured automatically.