PrismML Tracing

​

Prerequisites

​

Install

pip install arize-phoenix-otel openinference-instrumentation-openai openai

​

Setup

​

Run PrismML

from openai import OpenAI

# Point the OpenAI client at PrismML's local OpenAI-compatible endpoint.
client = OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="not-needed",
)

response = client.chat.completions.create(
    # The server accepts any model ID and serves the model it loaded.
    model="bonsai",
    messages=[
        {
            "role": "user",
            "content": "Why is the ocean salty? Answer in two sentences.",
        },
    ],
    # Bonsai models think before answering, which can use thousands of tokens.
    max_tokens=8192,
)

print(response.choices[0].message.content)

​

Observe

​

Common Issues

• Spans missing: register(..., auto_instrument=True) must run before your application sends requests, and openinference-instrumentation-openai must be installed in the same environment
• Connection refused or ConnectError to localhost:8080: The PrismML server is not running or is listening on a different port. Start it with PrismML’s server instructions
• Model not found: The server exposes the model it loaded. Use the model name returned by GET http://localhost:8080/v1/models, or use bonsai as shown in this guide
• Empty or cut-off answer: Thinking used up max_tokens before the model finished. PrismML recommends a max_tokens of 8,192 or more with thinking on

​

Resources