Skip to main content
TrueFoundry is an enterprise AI gateway: one OpenAI-compatible endpoint that fronts 1000+ models across 40+ providers (OpenAI, Anthropic, Bedrock, Vertex AI, Mistral, and more) with governance, budgets, and guardrails. The endpoint at https://gateway.truefoundry.ai mirrors OpenAI’s schema, so any OpenAI client works with base_url set to the gateway. Phoenix captures every gateway call via the openinference-instrumentation-openai package, the same instrumentor that covers OpenAI’s hosted API.
This guide instruments your application’s calls to the gateway, so the spans include your prompts, responses, and token usage as your code sees them.

Install

Setup

Add your TrueFoundry token as an environment variable. Use a Personal Access Token or Virtual Account Token from the Access section of TrueFoundry:
Connect your application to Phoenix with the register function:

Run TrueFoundry

The model name on the span is the resolved model the gateway routed to (for example, gpt-4.1-mini-2025-04-14), not the openai/gpt-4.1-mini identifier you passed.

Observe

Now that you have tracing set up, all invocations of the OpenAI client pointed at TrueFoundry will be streamed to your running Phoenix for observability and evaluation. Each call appears as a ChatCompletion LLM span with the prompt, response, model name, token usage, and latency.

Common Issues

  • Spans missing: register(..., auto_instrument=True) must run before your application sends requests, and openinference-instrumentation-openai must be installed in the same environment
  • 401 from TrueFoundry: Use a TrueFoundry token (Personal Access Token or Virtual Account Token from the Access section), not your OpenAI key. The OpenAI client sends it as a Bearer token
  • Model not found: Use the provider-account/model-name identifier from your gateway (for example, openai/gpt-4.1-mini). The prefix is the provider account name configured in your TrueFoundry workspace, not a fixed value. Copy the exact model ID from the Playground Code Snippets tab

Resources

TrueFoundry AI Gateway Documentation

OpenInference OpenAI Instrumentation