LiteLLM Tracing

LiteLLM allows developers to call all LLM APIs using the openAI format. LiteLLM Proxy is a proxy server to call 100+ LLMs in OpenAI format. Both are supported by this auto-instrumentation. Any calls made to the following functions will be automatically captured by this integration:

completion()
acompletion()
completion_with_retries()
embedding()
aembedding()
image_generation()
aimage_generation()

Install

pip install openinference-instrumentation-litellm "litellm<1.82.7"

Setup

Use the register function to connect your application to Phoenix:

from phoenix.otel import register

# configure the Phoenix tracer
tracer_provider = register(
  project_name="my-llm-app", # Default is 'default'
  auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)

Add any API keys needed by the models you are using with LiteLLM.

import os
os.environ["OPENAI_API_KEY"] = "PASTE_YOUR_API_KEY_HERE"

Run LiteLLM

You can now use LiteLLM as normal and calls will be traces in Phoenix.

import litellm
completion_response = litellm.completion(model="gpt-3.5-turbo",
                   messages=[{"content": "What's the capital of China?", "role": "user"}])
print(completion_response)

Observe

Traces should now be visible in Phoenix!

Resources

OpenInference Instrumentation

Developer Tools

MCP

LLM Providers

TypeScript

Python

Java

Platform

Sandboxes

Evaluation Integrations

LiteLLM Tracing

Install

Setup

Run LiteLLM

Observe

Resources

​Install

​Setup

​Run LiteLLM

​Observe

​Resources

Install

Setup

Run LiteLLM

Observe

Resources