vLLM Tracing

vLLM is an inference and serving engine for large language models. You can observe vLLM with Phoenix in two complementary ways:

Export vLLM server spans over OTLP with --otlp-traces-endpoint.
Trace your application calls to vLLM’s OpenAI-compatible server with openinference-instrumentation-openai.

Use both paths when possible. vLLM’s server-side OTLP spans show inference-server timing and request metadata. OpenInference OpenAI instrumentation captures the client-side prompt, response, token, model, and latency metadata for calls made through the OpenAI-compatible API.

Option 1: Export vLLM server spans over OTLP

vLLM defaults to OTLP/gRPC. Phoenix accepts OTLP/HTTP at <PHOENIX_COLLECTOR_ENDPOINT>/v1/traces, so set the exporter protocol before starting vLLM. This example sends spans to a local Phoenix at http://localhost:6006:

export OTEL_SERVICE_NAME="vllm-server"
export OTEL_RESOURCE_ATTRIBUTES="openinference.project.name=vllm-server"
export OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf

vllm serve <model-name-or-path> \
  --otlp-traces-endpoint "http://localhost:6006/v1/traces"

If your Phoenix instance has authentication enabled (for example, Phoenix Cloud), point --otlp-traces-endpoint at your instance’s /v1/traces URL and pass your API key as a header:

export OTEL_EXPORTER_OTLP_TRACES_HEADERS="authorization=Bearer <your-phoenix-api-key>"

Current vLLM releases bundle the core OpenTelemetry packages needed for server-side tracing. If you use an older or custom vLLM image and it errors on startup after you add --otlp-traces-endpoint, install matching OpenTelemetry exporter dependencies in the same environment or container image.

Option 2: Trace OpenAI-compatible client calls

Install

pip install arize-phoenix-otel openinference-instrumentation-openai openai

Setup

Connect your application to Phoenix with the register function:

Run vLLM

vLLM’s OpenAI-compatible server does not require an API key unless you start it with --api-key. The OpenAI client still requires a non-empty value, so the example uses EMPTY.

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="EMPTY",
)

response = client.chat.completions.create(
    model="<model-name-served-by-vllm>",
    messages=[
        {
            "role": "user",
            "content": "Name two signals that help debug inference latency.",
        }
    ],
)

print(response.choices[0].message.content)

Observe

Open Phoenix and select the my-llm-app project for client-side spans. Each call appears as a ChatCompletion LLM span with the prompt, response, model, token usage, and latency. If you also enabled vLLM server OTLP export, select the vllm-server project to see server-side inference and request-processing spans.

Common Issues

No client-side LLM spans: Make sure register(..., auto_instrument=True) runs before your OpenAI client sends requests
No vLLM server spans: Confirm OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf is set and --otlp-traces-endpoint points to your Phoenix collector’s /v1/traces path
Server spans appear in the default project: Confirm OTEL_RESOURCE_ATTRIBUTES includes openinference.project.name=vllm-server
Server spans lack prompt or response text: Use the OpenInference OpenAI client path as well. vLLM’s server export does not include the same application-level input and output detail

Developer Tools

LLM Providers

TypeScript

Python

Java

.NET

Platform

Vector Databases

Sandboxes

Evaluation Integrations

Coding Agents

Option 1: Export vLLM server spans over OTLP

Option 2: Trace OpenAI-compatible client calls

Install

Setup

Run vLLM

Observe

Common Issues

Resources

vLLM OpenTelemetry Example

vLLM OpenAI-Compatible Server

OpenInference OpenAI Instrumentation

​Option 1: Export vLLM server spans over OTLP

​Option 2: Trace OpenAI-compatible client calls

​Install

​Setup

​Run vLLM

​Observe

​Common Issues

​Resources

vLLM OpenTelemetry Example

vLLM OpenAI-Compatible Server

OpenInference OpenAI Instrumentation

Option 1: Export vLLM server spans over OTLP

Option 2: Trace OpenAI-compatible client calls

Install

Setup

Run vLLM

Observe

Common Issues

Resources