Featured Tutorials

Request or Contribute an Integration

Integration Types

Phoenix has a wide range of integrations. Generally these fall into a few categories:

1. Tracing integrations - where Phoenix will capture traces of applications built using a specific library.
2. Eval Model integrations - where Phoenix's eval python package will make calls to a specific model.
3. Eval Library integrations - where Phoenix traces can be evaluated using an outside eval library, instead of Phoenix's eval library, and visualized in Phoenix.

Each partner listing in this section contains integration docs and relevant tutorials.

AnthropicModel

class AnthropicModel(BaseModel):
    model: str = "claude-2.1"
    """The model name to use."""
    temperature: float = 0.0
    """What sampling temperature to use."""
    max_tokens: int = 256
    """The maximum number of tokens to generate in the completion."""
    top_p: float = 1
    """Total probability mass of tokens to consider at each step."""
    top_k: int = 256
    """The cutoff where the model no longer selects the words."""
    stop_sequences: List[str] = field(default_factory=list)
    """If the model encounters a stop sequence, it stops generating further tokens."""
    extra_parameters: Dict[str, Any] = field(default_factory=dict)
    """Any extra parameters to add to the request body (e.g., countPenalty for a21 models)"""
    max_content_size: Optional[int] = None
    """If you're using a fine-tuned model, set this to the maximum content size"""

Usage

In this section, we will showcase the methods and properties that our EvalModels have. First, instantiate your model from the. Once you've instantiated your model, you can get responses from the LLM by simply calling the model and passing a text string.

model = #Instantiate your Anthropic model here
model("Hello there, how are you?")
# Output: "As an artificial intelligence, I don't have feelings, 
#          but I'm here and ready to assist you. How can I help you today?"

GeminiModel

class GeminiModel:
    project: Optional[str] = None
    location: Optional[str] = None
    credentials: Optional["Credentials"] = None
    model: str = "gemini-pro"
    default_concurrency: int = 5
    temperature: float = 0.0
    max_tokens: int = 256
    top_p: float = 1
    top_k: int = 32

To authenticate with Gemini, you must pass either your credentials or a project, location pair. In the following example, we quickly instantiate the Gemini model as follows:

project = "my-project-id"
location = "us-central1" # as an example
model = GeminiModel(project=project, location=location)
model("Hello there, this is a tesst if you are working?")
# Output: "Hello world, I am working!"

Featured Tutorials

class LiteLLMModel(BaseEvalModel):
    model: str = "gpt-3.5-turbo"
    """The model name to use."""
    temperature: float = 0.0
    """What sampling temperature to use."""
    max_tokens: int = 256
    """The maximum number of tokens to generate in the completion."""
    top_p: float = 1
    """Total probability mass of tokens to consider at each step."""
    num_retries: int = 6
    """Maximum number to retry a model if an RateLimitError, OpenAIError, or
    ServiceUnavailableError occurs."""
    request_timeout: int = 60
    """Maximum number of seconds to wait when retrying."""
    model_kwargs: Dict[str, Any] = field(default_factory=dict)
    """Model specific params"""

Here is an example of how to initialize LiteLLMModel for llama3 using ollama.

import os

from phoenix.evals import LiteLLMModel

os.environ["OLLAMA_API_BASE"] = "http://localhost:11434"

model = LiteLLMModel(model="ollama/llama3")

Launch Phoenix

Install

pip install openinference-instrumentation-mistralai mistralai

Setup

Set the MISTRAL_API_KEY environment variable to authenticate calls made using the SDK.

export MISTRAL_API_KEY=[your_key_here]

Connect to your Phoenix instance using the register function.

from phoenix.otel import register

# configure the Phoenix tracer
tracer_provider = register(
  project_name="my-llm-app", # Default is 'default'
  auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)

Run Mistral

import os

from mistralai import Mistral
from mistralai.models import UserMessage

api_key = os.environ["MISTRAL_API_KEY"]
model = "mistral-tiny"

client = Mistral(api_key=api_key)

chat_response = client.chat.complete(
    model=model,
    messages=[UserMessage(content="What is the best French cheese?")],
)
print(chat_response.choices[0].message.content)

Observe

Now that you have tracing setup, all invocations of Mistral (completions, chat completions, embeddings) will be streamed to your running Phoenix for observability and evaluation.

Resources

Featured Tutorials

Featured Tutorials

Launch Phoenix

We have several code samples below on different ways to integrate with OpenAI, based on how you want to use Phoenix.

Install

pip install openinference-instrumentation-openai openai

Setup

Add your OpenAI API key as an environment variable:

export OPENAI_API_KEY=[your_key_here]

Use the register function to connect your application to Phoenix:

from phoenix.otel import register

# configure the Phoenix tracer
tracer_provider = register(
  project_name="my-llm-app", # Default is 'default'
  auto_instrument=True # Auto-instrument your app based on installed dependencies
)

Run OpenAI

import openai

client = openai.OpenAI()
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Write a haiku."}],
)
print(response.choices[0].message.content)

Observe

Now that you have tracing setup, all invocations of OpenAI (completions, chat completions, embeddings) will be streamed to your running Phoenix for observability and evaluation.

Resources

Featured Tutorials

Featured Tutorials

Featured Tutorials

Featured Tutorials

Featured Tutorials

Website: mongodb.com

Phoenix can be used to trace and evaluate applications that use MongoDB Atlas as a vector database.

Featured Tutorials

boto3 provides Python bindings to AWS services, including Bedrock, which provides access to a number of foundation models. Calls to these models can be instrumented using OpenInference, enabling OpenTelemetry-compliant observability of applications built using these models. Traces collected using OpenInference can be viewed in Phoenix.

OpenInference Traces collect telemetry data about the execution of your LLM application. Consider using this instrumentation to understand how a Bedrock-managed models are being called inside a complex system and to troubleshoot issues such as extraction and response synthesis.

Launch Phoenix

Install

Setup

Connect to your Phoenix instance using the register function.

After connecting to your Phoenix server, instrument boto3 prior to initializing a bedrock-runtime client. All clients created after instrumentation will send traces on all calls to invoke_model.

Run Bedrock

From here you can run Bedrock as normal

Observe

Now that you have tracing setup, all calls to invoke_model will be streamed to your running Phoenix for observability and evaluation.

Resources

BedrockModel

To Authenticate, the following code is used to instantiate a session and the session is used with Phoenix Evals

Amazon Bedrock Agents allow you to easily define, deploy, and manage agents on your AWS infrastructure. Traces on invocations of these agents can be captured using OpenInference and viewed in Phoenix.

This instrumentation will capture data on LLM calls, action group invocations (as tools), knowledgebase lookups, and more.

Launch Phoenix

Install

Setup

Connect to your Phoenix instance using the register function.

After connecting to your Phoenix server, instrument boto3 prior to initializing a bedrock-runtime client. All clients created after instrumentation will send traces on all calls to invoke_model, invoke_agent, and their streaming variations.

Run Bedrock Agents

From here you can run Bedrock as normal

Observe

Now that you have tracing setup, all calls will be streamed to your running Phoenix for observability and evaluation.

Resources

Launch Phoenix

Install

Setup

Use the register function to connect your application to Phoenix.

Observe

Now that you have tracing setup, all GenAI SDK requests will be streamed to Phoenix for observability and evaluation.

Launch Phoenix

Install

Setup

Use the register function to connect your application to Phoenix:

Run VertexAI

Observe

Now that you have tracing setup, all invocations of Vertex models will be streamed to your running Phoenix for observability and evaluation.

Resources

MistralAIModel

Usag

VertexAI

To authenticate with VertexAI, you must pass either your credentials or a project, location pair. In the following example, we quickly instantiate the VertexAI model as follows:

Featured Tutorials

Featured Tutorials

Analyzing and troubleshooting what happens under the hood can be challenging without proper insights. By integrating your Flowise application with Phoenix, you can monitor traces and gain robust observability into your chatflows and agentflows.

Viewing Flowise traces in Phoenix

1. Access Configurations: Navigate to settings in your chatflow or agentflow and find configurations.

1. Connect to Phoenix: Go to the Analyze Chatflow tab and configure your application with Phoenix. Get your API key from your Phoenix instance to create your credentials. Be sure to name your project and confirm that the Phoenix toggle is enabled before saving.

1. View Traces: In Phoenix, you will find your project under the Projects tab. Click into this to view and analyze traces as you test your application.

1. Store and Experiment: Optionally, you can also filter traces, store traces in a dataset to run experiments, analyze patterns, and optimize your workflows over time.

Phoenix can be used to trace and evaluate applications that use Weaviate as a vector database.

Examples

Phoenix can be used to trace and evaluate applications that use Zilliz or Milvus as a vector database.

Examples:

Featured Tutorials

Pull Langflow Repo

Navigate to the Langflow GitHub repo and pull the project down

Create .env file

Navigate to the repo and create a .env file with all the Arize Phoenix variables.

You can use the .env.example as a template to create the .env file

Add the following environment variable to the .env file

Start Docker Desktop

Start Docker Desktop, build the images, and run the container (this will take around 10 minutes the first time) Go into your terminal into the Langflow directory and run the following commands

Go to Hosted Langflow UI

Create a Flow

In this example, we'll use Simple Agent for this tutorial

Add your OpenAI Key to the Agent component in Langflow

Go into the Playground and run the Agent

Go to Arize Phoenix

Navigate to your project name (should match the name of of your Langflow Agent name)

Inspect Traces

AgentExecutor Trace is Arize Phoenix instrumentation to capture what's happening with the LangChain being ran during the Langflow components

The other UUID trace is the native Langflow tracing.

Phoenix can be used to trace and evaluate applications that use Qdrant as a vector database.

Examples

Phoenix provides seamless observability and tracing for Agno agents through the OpenInference instrumentation package. This integration automatically captures agent interactions, tool usage, reasoning steps, and multi-agent conversations, giving you complete visibility into your Agno applications. Monitor performance, debug issues, and evaluate agent behavior in real-time as your agents execute complex workflows and collaborate in teams.

Agno is a lightweight, high-performance Python framework for building AI agents with tools, memory, and reasoning capabilities. It enables developers to create autonomous agents that can perform complex tasks, access knowledge bases, and collaborate in multi-agent teams. With support for 23+ model providers and lightning-fast performance (~3μs instantiation), Agno is designed for production-ready AI applications.

Key Features

• Model Agnostic: Connect to OpenAI, Anthropic, Google, and 20+ other providers

• Lightning Fast: Agents instantiate in ~3μs with minimal memory footprint

• Built-in Reasoning: First-class support for chain-of-thought and reasoning models

• Multi-Modal: Native support for text, image, audio, and video processing

• Agentic RAG: Advanced retrieval-augmented generation with hybrid search

• Multi-Agent Teams: Coordinate multiple agents for complex workflows

• Production Ready: Pre-built FastAPI routes and monitoring capabilities

Launch Phoenix

Install

pip install openinference-instrumentation-agno agno

Setup

Use the register function to connect your application to Phoenix:

from phoenix.otel import register

# configure the Phoenix tracer
tracer_provider = register(
  project_name="my-llm-app", # Default is 'default'
  auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)

Run Agno

from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.tools.duckduckgo import DuckDuckGoTools

agent = Agent(
    model=OpenAIChat(id="gpt-4o-mini"),
    tools=[DuckDuckGoTools()],
    markdown=True,
    debug_mode=True,
)

agent.run("What is currently trending on Twitter?")

Observe

Now that you have tracing setup, all invocations of Agno agents will be streamed to Phoenix for observability and evaluation.

Resources

Install

npm install --save @arizeai/openinference-instrumentation-openai

Setup

To load the OpenAI instrumentation, specify it in the registerInstrumentations call along with any additional instrumentation you wish to enable.

import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node";
import {
  OpenAIInstrumentation,
} from "@arizeai/openinference-instrumentation-openai";
import { registerInstrumentations } from "@opentelemetry/instrumentation";

const provider = new NodeTracerProvider();
provider.register();

registerInstrumentations({
  instrumentations: [new OpenAIInstrumentation()],
});

Support

Instrumentation version >1.0.0 supports both attribute masking and context attribute propagation to spans.

Resources

Launch Phoenix

Install

pip install openinference-instrumentation-groq groq

Setup

Connect to your Phoenix instance using the register function.

from phoenix.otel import register

# configure the Phoenix tracer
tracer_provider = register(
  project_name="my-llm-app", # Default is 'default'
  auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)

Run Groq

A simple Groq application that is now instrumented

import os
from groq import Groq

client = Groq(
    # This is the default and can be omitted
    api_key=os.environ.get("GROQ_API_KEY"),
)

chat_completion = client.chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": "Explain the importance of low latency LLMs",
        }
    ],
    model="mixtral-8x7b-32768",
)
print(chat_completion.choices[0].message.content)

Observe

Now that you have tracing setup, all invocations of pipelines will be streamed to your running Phoenix for observability and evaluation.

Resources:

Featured Tutorials

This module provides automatic instrumentation for LangChain.js, more specifically, the @langchain/core module. which may be used in conjunction with @opentelemetry/sdk-trace-node.

Install

npm install --save @arizeai/openinference-instrumentation-langchain

Setup

To load the LangChain instrumentation, manually instrument the @langchain/core/callbacks/manager module. The callbacks manager must be manually instrumented due to the non-traditional module structure in @langchain/core. Additional instrumentations can be registered as usual in the registerInstrumentations function.

import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node";
import { 
  LangChainInstrumentation 
} from "@arizeai/openinference-instrumentation-langchain";
import * as CallbackManagerModule from "@langchain/core/callbacks/manager";

const provider = new NodeTracerProvider();
provider.register();

const lcInstrumentation = new LangChainInstrumentation();
// LangChain must be manually instrumented as it doesn't have 
// a traditional module structure
lcInstrumentation.manuallyInstrument(CallbackManagerModule);

Support

Instrumentation version >1.0.0 supports both attribute masking and context attribute propagation to spans.

Resources

Install

pip install openinference-instrumentation-anthropic anthropic

Setup

Use the register function to connect your application to Phoenix:

from phoenix.otel import register

# configure the Phoenix tracer
tracer_provider = register(
  project_name="my-llm-app", # Default is 'default'
  auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)

Run Anthropic

A simple Anthropic application that is now instrumented

import anthropic

client = anthropic.Anthropic()

message = client.messages.create(
    model="claude-3-5-sonnet-20240620",
    max_tokens=1000,
    temperature=0,
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "Why is the ocean salty?"
                }
            ]
        }
    ]
)
print(message.content)

Observe

Now that you have tracing setup, all invocations of pipelines will be streamed to your running Phoenix for observability and evaluation.

Resources:

Phoenix can be used to trace and evaluate applications that use Pinecone as a vector database.

Examples

Launch Phoenix

Install

Setup

Set up the OpenTelemetry endpoint to point to Phoenix and use Prompt flow's setup_exporter_from_environ to start tracing any further flows and LLM calls.

Run PromptFlow

Observe

You should see the spans render in Phoenix as shown in the below screenshots.

Resources

1. Launch Phoenix

We have several code samples below on different ways to integrate with OpenAI, based on how you want to use Phoenix.

2. Install

pip install openinference-instrumentation-openai-agents openai-agents

3. Set up Tracing

Add your OpenAI API key as an environment variable:

export OPENAI_API_KEY=[your_key_here]

Use the register function to connect your application to Phoenix:

from phoenix.otel import register

# configure the Phoenix tracer
tracer_provider = register(
  project_name="agents", # Default is 'default'
  auto_instrument=True # Auto-instrument your app based on installed dependencies
)

Run your agentscode.

from agents import Agent, Runner

agent = Agent(name="Assistant", instructions="You are a helpful assistant")
result = Runner.run_sync(agent, "Write a haiku about recursion in programming.")
print(result.final_output)

View your traces in Phoenix.

Resources

Any calls made to the following functions will be automatically captured by this integration:

• completion()

• acompletion()

• completion_with_retries()

• embedding()

• aembedding()

• image_generation()

• aimage_generation()

Launch Phoenix

Install

pip install openinference-instrumentation-litellm litellm

Setup

Use the register function to connect your application to Phoenix:

from phoenix.otel import register

# configure the Phoenix tracer
tracer_provider = register(
  project_name="my-llm-app", # Default is 'default'
  auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)

Add any API keys needed by the models you are using with LiteLLM.

import os
os.environ["OPENAI_API_KEY"] = "PASTE_YOUR_API_KEY_HERE"

Run LiteLLM

You can now use LiteLLM as normal and calls will be traces in Phoenix.

import litellm
completion_response = litellm.completion(model="gpt-3.5-turbo",
                   messages=[{"content": "What's the capital of China?", "role": "user"}])
print(completion_response)

Observe

Traces should now be visible in Phoenix!

Resources

OpenAIModel

class OpenAIModel:
    api_key: Optional[str] = field(repr=False, default=None)
    """Your OpenAI key. If not provided, will be read from the environment variable"""
    organization: Optional[str] = field(repr=False, default=None)
    """
    The organization to use for the OpenAI API. If not provided, will default
    to what's configured in OpenAI
    """
    base_url: Optional[str] = field(repr=False, default=None)
    """
    An optional base URL to use for the OpenAI API. If not provided, will default
    to what's configured in OpenAI
    """
    model: str = "gpt-4"
    """Model name to use. In of azure, this is the deployment name such as gpt-35-instant"""
    temperature: float = 0.0
    """What sampling temperature to use."""
    max_tokens: int = 256
    """The maximum number of tokens to generate in the completion.
    -1 returns as many tokens as possible given the prompt and
    the models maximal context size."""
    top_p: float = 1
    """Total probability mass of tokens to consider at each step."""
    frequency_penalty: float = 0
    """Penalizes repeated tokens according to frequency."""
    presence_penalty: float = 0
    """Penalizes repeated tokens."""
    n: int = 1
    """How many completions to generate for each prompt."""
    model_kwargs: Dict[str, Any] = field(default_factory=dict)
    """Holds any model parameters valid for `create` call not explicitly specified."""
    batch_size: int = 20
    """Batch size to use when passing multiple documents to generate."""
    request_timeout: Optional[Union[float, Tuple[float, float]]] = None
    """Timeout for requests to OpenAI completion API. Default is 600 seconds."""

To authenticate with OpenAI you will need, at a minimum, an API key. The model class will look for it in your environment, or you can pass it via argument as shown above. In addition, you can choose the specific name of the model you want to use and its configuration parameters. The default values specified above are common default values from OpenAI. Quickly instantiate your model as follows:

model = OpenAI()
model("Hello there, this is a test if you are working?")
# Output: "Hello! I'm working perfectly. How can I assist you today?"

Azure OpenAI

The code snippet below shows how to initialize OpenAIModel for Azure:

model = OpenAIModel(
    model="gpt-35-turbo-16k",
    azure_endpoint="https://arize-internal-llm.openai.azure.com/",
    api_version="2023-09-15-preview",
)

Azure OpenAI supports specific options:

api_version: str = field(default=None)
"""
The verion of the API that is provisioned
https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#rest-api-versioning
"""
azure_endpoint: Optional[str] = field(default=None)
"""
The endpoint to use for azure openai. Available in the azure portal.
https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal#create-a-resource
"""
azure_deployment: Optional[str] = field(default=None)
azure_ad_token: Optional[str] = field(default=None)
azure_ad_token_provider: Optional[Callable[[], str]] = field(default=None)

Installation

npm install --save @arizeai/openinference-instrumentation-beeai beeai-framework

npm install --save @opentelemetry/sdk-node @opentelemetry/exporter-trace-otlp-http @opentelemetry/semantic-conventions @arizeai/openinference-semantic-conventions

Usage

To instrument your application, import and enable BeeAIInstrumentation

1. Create the instrumentation.js file:

// instrumentation.js
import { NodeSDK, node, resources } from "@opentelemetry/sdk-node";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-proto";
import { ATTR_SERVICE_NAME } from "@opentelemetry/semantic-conventions";
import { SEMRESATTRS_PROJECT_NAME } from "@arizeai/openinference-semantic-conventions";
import { BeeAIInstrumentation } from "@arizeai/openinference-instrumentation-beeai";
import * as beeaiFramework from "beeai-framework";

// Initialize Instrumentation Manually
const beeAIInstrumentation = new BeeAIInstrumentation();

const provider = new NodeSDK({
  resource: new resources.Resource({
    [ATTR_SERVICE_NAME]: "beeai",
    [SEMRESATTRS_PROJECT_NAME]: "beeai-project",
  }),
  spanProcessors: [
    new node.SimpleSpanProcessor(
      new OTLPTraceExporter({
        url: "http://localhost:6006/v1/traces",
      }),
    ),
  ],
  instrumentations: [beeAIInstrumentation],
});

await provider.start();

// Manually Patch BeeAgent (This is needed when the module is not loaded via require (commonjs))
console.log("🔧 Manually instrumenting BeeAgent...");
beeAIInstrumentation.manuallyInstrument(beeaiFramework);
console.log("✅ BeeAgent manually instrumented.");

// eslint-disable-next-line no-console
console.log("👀 OpenInference initialized");

1. Import the library and call the BeeAI framework

import "./instrumentation";
import { BeeAgent } from "beeai-framework/agents/bee/agent";
import { TokenMemory } from "beeai-framework/memory/tokenMemory";
import { DuckDuckGoSearchTool } from "beeai-framework/tools/search/duckDuckGoSearch";
import { OpenMeteoTool } from "beeai-framework/tools/weather/openMeteo";
import { OllamaChatModel } from "beeai-framework/adapters/ollama/backend/chat";

const llm = new OllamaChatModel("llama3.1");
const agent = new BeeAgent({
  llm,
  memory: new TokenMemory(),
  tools: [new DuckDuckGoSearchTool(), new OpenMeteoTool()],
});

const response = await agent.run({
  prompt: "What's the current weather in Berlin?",
});

console.log(`Agent 🤖 : `, response.result.text);

Launch Phoenix

Install

pip install openinference-instrumentation-llama_index

Setup

Initialize the LlamaIndexInstrumentor before your application code. This instrumentor will trace both LlamaIndex Workflows calls, as well as calls to the general LlamaIndex package.

from openinference.instrumentation.llama_index import LlamaIndexInstrumentor
from phoenix.otel import register

tracer_provider = register()
LlamaIndexInstrumentor().instrument(tracer_provider=tracer_provider)

Run LlamaIndex Workflows

By instrumenting LlamaIndex, spans will be created whenever an agent is invoked and will be sent to the Phoenix server for collection.

Observe

Now that you have tracing setup, all invocations of chains will be streamed to your running Phoenix for observability and evaluation.

Resources

Launch Phoenix

Install

pip install openinference-instrumentation-instructor instructor

Be sure you also install the OpenInference library for the underlying model you're using along with Instructor. For example, if you're using OpenAI calls directly, you would also add: openinference-instrumentation-openai

Setup

Connect to your Phoenix instance using the register function.

from phoenix.otel import register

# configure the Phoenix tracer
tracer_provider = register(
  project_name="my-llm-app", # Default is 'default'
  auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)

Run Instructor

From here you can use instructor as normal.

import instructor
from pydantic import BaseModel
from openai import OpenAI


# Define your desired output structure
class UserInfo(BaseModel):
    name: str
    age: int


# Patch the OpenAI client
client = instructor.from_openai(OpenAI())

# Extract structured data from natural language
user_info = client.chat.completions.create(
    model="gpt-3.5-turbo",
    response_model=UserInfo,
    messages=[{"role": "user", "content": "John Doe is 30 years old."}],
)

print(user_info.name)
#> John Doe
print(user_info.age)
#> 30

Observe

Now that you have tracing setup, all invocations of your underlying model (completions, chat completions, embeddings) and instructor triggers will be streamed to your running Phoenix for observability and evaluation.

AutoGen is an agent framework from Microsoft that allows for complex Agent creation. It is unique in its ability to create multiple agents that work together.

The AutoGen Agent framework allows creation of multiple agents and connection of those agents to work together to accomplish tasks.

Launch Phoenix

pip install arize-phoenix-otel

import os

# Add Phoenix API Key for tracing
PHOENIX_API_KEY = "ADD YOUR API KEY"
os.environ["PHOENIX_CLIENT_HEADERS"] = f"api_key={PHOENIX_API_KEY}"
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "https://app.phoenix.arize.com"

pip install arize-phoenix
phoenix serve

pip install arize-phoenix-otel

import os

os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"

docker pull arizephoenix/phoenix:latest

docker run -p 6006:6006 arizephoenix/phoenix:latest

pip install arize-phoenix-otel

import os

os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"

pip install arize-phoenix

import phoenix as px
px.launch_app()

Install

Phoenix instruments Autogen by instrumenting the underlying model library it's using. If your agents are set up to call OpenAI, use our OpenAI instrumentor per the example below.

If your agents are using a different model, be sure to instrument that model instead by installing its respective OpenInference library.

pip install openinference-instrumentation-openai arize-phoenix-otel arize-phoenix

Setup

Connect to your Phoenix instance using the register function.

from phoenix.otel import register

# configure the Phoenix tracer
tracer_provider = register(
  project_name="my-llm-app", # Default is 'default'
  auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)

Run Autogen

From here you can use Autogen as normal, and Phoenix will automatically trace any model calls made.

Observe

The Phoenix support is simple in its first incarnation but allows for capturing all of the prompt and responses that occur under the framework between each agent.

The individual prompt and responses are captured directly through OpenAI calls. If you're using a different underlying model provider than OpenAI, instrument your application using the respective instrumentor instead.

Resources:

Launch Phoenix

Install

pip install openinference-instrumentation-langchain langchain_openai

Setup

Use the register function to connect your application to Phoenix:

from phoenix.otel import register

# configure the Phoenix tracer
tracer_provider = register(
  project_name="my-llm-app", # Default is 'default'
  auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)

Run LangChain

By instrumenting LangChain, spans will be created whenever a chain is run and will be sent to the Phoenix server for collection.

from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI

prompt = ChatPromptTemplate.from_template("{x} {y} {z}?").partial(x="why is", z="blue")
chain = prompt | ChatOpenAI(model_name="gpt-3.5-turbo")
chain.invoke(dict(y="sky"))

Observe

Now that you have tracing setup, all invocations of chains will be streamed to your running Phoenix for observability and evaluation.

Resources

Phoenix MCP Server supports:

• Prompts Management: Create, list, update, and iterate on prompts

• Datasets: Explore datasets, and synthesize new examples

• Experiments: Pull experiment results and visualize them with the help of an LLM

Connecting the Phoenix MCP Server

Using the Phoenix MCP Server

The MCP server can be used to interact with Prompts, Experiments, and Datasets. It can be used to retrieve information about each item, and can create and update Prompts.

Some good questions to try:

1. What prompts do I have in Phoenix?

2. Create a new prompt in Phoenix that classifies user intent

3. Update my classification prompt in Phoenix with these new options

4. Summarize the Phoenix experiments run on my agent inputs dataset

5. Visualize the results of my jailbreak dataset experiments in Phoenix

Hoping to see additional functionality?

OpenInference Vercel

Note: This package requires you to be using the Vercel AI SDK version 3.3 or higher.

Installation

npm install --save @arizeai/openinference-vercel

You will also need to install OpenTelemetry and Vercel packages to your project.

npm i @opentelemetry/api @vercel/otel @opentelemetry/exporter-trace-otlp-proto @arizeai/openinference-semantic-conventions

Usage

To process your Vercel AI SDK Spans add a OpenInferenceSimpleSpanProcessor or OpenInferenceBatchSpanProcessor to your OpenTelemetry configuration.

import { registerOTel } from "@vercel/otel";
import { diag, DiagConsoleLogger, DiagLogLevel } from "@opentelemetry/api";
import {
  isOpenInferenceSpan,
  OpenInferenceSimpleSpanProcessor,
} from "@arizeai/openinference-vercel";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-proto";
import { SEMRESATTRS_PROJECT_NAME } from "@arizeai/openinference-semantic-conventions";

// For troubleshooting, set the log level to DiagLogLevel.DEBUG
// This is not required and should not be added in a production setting
diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.DEBUG);

export function register() {
  registerOTel({
    serviceName: "phoenix-next-app",
    attributes: {
      // This is not required but it will allow you to send traces to a specific 
      // project in phoenix
      [SEMRESATTRS_PROJECT_NAME]: "your-next-app",
    },
    spanProcessors: [
      new OpenInferenceSimpleSpanProcessor({
        exporter: new OTLPTraceExporter({
          headers: {
            // API key if you're sending it to Phoenix
            api_key: process.env["PHOENIX_API_KEY"],
          },
          url:
            process.env["PHOENIX_COLLECTOR_ENDPOINT"] ||
            "https://app.phoenix.arize.com/v1/traces",
        }),
        spanFilter: (span) => {
          // Only export spans that are OpenInference to remove non-generative spans
          // This should be removed if you want to export all spans
          return isOpenInferenceSpan(span);
        },
      }),
    ],
  });
}

Now enable telemetry in your AI SDK calls by setting the experimental_telemetry parameter to true.

const result = await generateText({
  model: openai("gpt-4-turbo"),
  prompt: "Write a short story about a cat.",
  experimental_telemetry: { isEnabled: true },
});

Examples

Launch Phoenix

pip install arize-phoenix-otel

import os

# Add Phoenix API Key for tracing
PHOENIX_API_KEY = "ADD YOUR API KEY"
os.environ["PHOENIX_CLIENT_HEADERS"] = f"api_key={PHOENIX_API_KEY}"
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "https://app.phoenix.arize.com"

pip install arize-phoenix
phoenix serve

pip install arize-phoenix-otel

import os

os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"

docker pull arizephoenix/phoenix:latest

docker run -p 6006:6006 arizephoenix/phoenix:latest

pip install arize-phoenix-otel

import os

os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"

pip install arize-phoenix

import phoenix as px
px.launch_app()

Install

pip install openinference-instrumentation-guardrails guardrails-ai

Setup

Connect to your Phoenix instance using the register function.

from phoenix.otel import register

# configure the Phoenix tracer
tracer_provider = register(
  project_name="my-llm-app", # Default is 'default'
  auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)

Run Guardrails

From here, you can run Guardrails as normal:

from guardrails import Guard
from guardrails.hub import TwoWords
import openai

guard = Guard().use(
    TwoWords(),
)
response = guard(
    llm_api=openai.chat.completions.create,
    prompt="What is another name for America?",
    model="gpt-3.5-turbo",
    max_tokens=1024,
)

print(response)

Observe

Now that you have tracing setup, all invocations of underlying models used by Guardrails (completions, chat completions, embeddings) will be streamed to your running Phoenix for observability and evaluation. Additionally, Guards will be present as a new span kind in Phoenix.

Resources

Launch Phoenix

Setup

Install packages:

Initialize OpenTelemetry tracing for your Mastra application:

From here you can use Mastra as normal. All agents, workflows, and tool calls will be automatically traced.

Example Agent Walkthrough

Here is a full project example to get you started:

Launch Phoenix using one of the methods above

The rest of this tutorial will assume you are running Phoenix locally on the default localhost:6006 port.

Create a new Mastra project

Connect to Phoenix

Add the OpenInference telemetry code to your index.js file. The complete file should now look like this:

Run the Agent

View your Traces in Phoenix

What Gets Traced

The Mastra instrumentation automatically captures:

• Agent Executions: Complete agent runs including instructions, model calls, and responses

• Workflow Steps: Individual step executions within workflows, including inputs, outputs, and timing

• Tool Calls: Function calls made by agents, including parameters and results

• LLM Interactions: All model calls with prompts, responses, token usage, and metadata

• RAG Operations: Vector searches, document retrievals, and embedding generations

• Memory Operations: Agent memory reads and writes

• Error Handling: Exceptions and error states in your AI pipeline

Trace Attributes

Phoenix will capture detailed attributes for each trace:

• Agent Information: Agent name, instructions, model configuration

• Workflow Context: Workflow name, step IDs, execution flow

• Tool Metadata: Tool names, parameters, execution results

• Model Details: Model name, provider, token usage, response metadata

• Performance Metrics: Execution time, token counts, costs

• User Context: Session IDs, user information (if provided)

You can view all of this information in the Phoenix UI to debug issues, optimize performance, and understand your application's behavior.

Launch Phoenix

Install

Setup

Use the register function to connect your application to Phoenix:

Run Portkey

By instrumenting Portkey, spans will be created whenever requests are made through the AI Gateway and will be sent to the Phoenix server for collection.

Basic Usage with OpenAI

Using Portkey SDK Directly

Observe

Now that you have tracing setup, all requests through Portkey's AI Gateway will be streamed to your running Phoenix instance for observability and evaluation. You'll be able to see:

• Request/Response Traces: Complete visibility into LLM interactions

• Routing Decisions: Which provider was selected and why

• Fallback Events: When and why fallbacks were triggered

• Cache Performance: Hit/miss rates and response times

• Cost Tracking: Token usage and costs across providers

• Latency Metrics: Response times for each provider and route

Resources

This guide will walk you through the process of creating and evaluating agents using Ragas and Arize Phoenix. We'll cover the following steps:

• Build a customer support agent with the OpenAI Agents SDK

• Trace agent activity to monitor interactions

• Generate a benchmark dataset for performance analysis

• Evaluate agent performance using Ragas

We will walk through the key steps in the documentation below. Check out the full tutorial here:

Creating the Agent

Here we've setup a basic agent that can solve math problems. We have a function tool that can solve math equations, and an agent that can use this tool. We'll use the Runner class to run the agent and get the final output.

Evaluating the Agent

Agents can go awry for a variety of reasons. We can use Ragas to evaluate whether the agent responded correctly. Two Ragas measurements help with this:

1. Tool Call Accuracy - Did our agent choose the right tool with the right arguments?

2. Agent Goal Accuracy - Did our agent accomplish the stated goal and get to the right outcome?

We'll import both metrics we're measuring from Ragas, and use the multi_turn_ascore(sample) to get the results. The AgentGoalAccuracyWithReference metric compares the final output to the reference to see if the goal was accomplished. The ToolCallAccuracy metric compares the tool call to the reference tool call to see if the tool call was made correctly.

In the notebook, we also define the helper function conversation_to_ragas_sample which converts the agent messages into a format that Ragas can use.

The following code snippets define our task function and evaluators.

Run the Experiment

Once we've generated a dataset of questions, we can use our experiments feature to track changes across models, prompts, parameters for the agent.

Finally, we run our experiment and view the results in Phoenix.

Phoenix makes your DSPy applications observable by visualizing the underlying structure of each call to your compiled DSPy module.

Launch Phoenix

Install

Setup

Connect to your Phoenix instance using the register function.

Run DSPy

Now run invoke your compiled DSPy module. Your traces should appear inside of Phoenix.

Observe

Now that you have tracing setup, all predictions will be streamed to your running Phoenix for observability and evaluation.

Resources

Ensuring the reliability and accuracy of LLM-generated responses is a critical challenge for production AI systems. Poor-quality training data, ambiguous labels, and untrustworthy outputs can degrade model performance and lead to unreliable results.

This guide demonstrates how to integrate Cleanlab’s Trustworthy Language Model (TLM) with Phoenix to systematically identify and improve low-quality LLM responses. By leveraging TLM for automated data quality assessment and Phoenix for response analysis, you can build more robust and trustworthy AI applications.

Specifically, this tutorial will walk through:

• Evaluating LLM-generated responses for trustworthiness.

• Using Cleanlab TLM to score and flag untrustworthy responses.

• Leveraging Phoenix for tracing and visualizing response evaluations.

Key Implementation Steps for generating evals w/ TLM

1. Install Dependencies, Set up API Keys, Obtain LLM Reponses + Trace in Phoenix

2. Download Trace Dataset

1. Prep data from trace dataset

1. Setup TLM & Evaluate each pair

1. Upload Evals to Phoenix

Check out the full tutorial here:

Launch Phoenix

Install

Install the OpenInference Langchain library before your application code. Our LangChainInstrumentor works for both standard LangChain applications and for LangGraph agents.

Setup

Use the register function to connect your application to Phoenix:

Run LangGraph

By instrumenting LangGraph, spans will be created whenever an agent is invoked and will be sent to the Phoenix server for collection.

Observe

Now that you have tracing setup, all invocations of chains will be streamed to your running Phoenix for observability and evaluation.

Resources