# model = Instantiate your MistralAIModel here
model("Hello there, how are you?")
# Output: "As an artificial intelligence, I don't have feelings,
# but I'm here and ready to assist you. How can I help you today?"
VertexAI Evals
Configure and run VertexAI for evals
VertexAI
Need to install the extra dependencygoogle-cloud-aiplatform>=1.33.0
To authenticate with VertexAI, you must pass either your credentials or a project, location pair. In the following example, we quickly instantiate the VertexAI model as follows:
project = "my-project-id"
location = "us-central1" # as an example
model = VertexAIModel(project=project, location=location)
model("Hello there, this is a tesst if you are working?")
# Output: "Hello world, I am working!"
Amazon Bedrock
Amazon Bedrock is a managed service that provides access to top AI models for building scalable applications.
Featured Tutorials
Overview
Request or Contribute an Integration
Integration Types
Phoenix has a wide range of integrations. Generally these fall into a few categories:
Tracing integrations - where Phoenix will capture traces of applications built using a specific library.
Eval Model integrations - where Phoenix's eval python package will make calls to a specific model.
Eval Library integrations - where Phoenix traces can be evaluated using an outside eval library, instead of Phoenix's eval library, and visualized in Phoenix.
Each partner listing in this section contains integration docs and relevant tutorials.
Google Gen AI
Google GenAI is a suite of AI tools and models from Google Cloud, designed to help businesses build, deploy, and scale AI applications.
BeeAI
BeeAI is an open-source platform that enables developers to discover, run, and compose AI agents from any framework, facilitating the creation of interoperable multi-agent systems
Flowise
Flowise is a low-code platform for building customized chatflows and agentflows.
VertexAI
Vertex AI is a fully managed platform by Google Cloud for building, deploying, and scaling machine learning models.
Mastra
Mastra is an open-source TypeScript AI agent framework designed for building production-ready AI applications with agents, workflows, RAG, and observability
LangFlow
Langflow is an open-source visual framework that enables developers to rapidly design, prototype, and deploy custom applications powered by large language models (LLMs)
Amazon Bedrock Tracing
Instrument LLM calls to AWS Bedrock via the boto3 client using the BedrockInstrumentor
boto3 provides Python bindings to AWS services, including Bedrock, which provides access to a number of foundation models. Calls to these models can be instrumented using OpenInference, enabling OpenTelemetry-compliant observability of applications built using these models. Traces collected using OpenInference can be viewed in Phoenix.
OpenInference Traces collect telemetry data about the execution of your LLM application. Consider using this instrumentation to understand how a Bedrock-managed models are being called inside a complex system and to troubleshoot issues such as extraction and response synthesis.
Launch Phoenix
Install
Setup
Connect to your Phoenix instance using the register function.
After connecting to your Phoenix server, instrument boto3 prior to initializing a bedrock-runtime client. All clients created after instrumentation will send traces on all calls to invoke_model.
Run Bedrock
From here you can run Bedrock as normal
Observe
Now that you have tracing setup, all calls to invoke_model will be streamed to your running Phoenix for observability and evaluation.
Resources
LiteLLM
LiteLLM is an open-source platform that provides a unified interface to manage and access over 100 LLMs from various providers.
OpenAI
OpenAI provides state-of-the-art LLMs for natural language understanding and generation.
Featured Tutorials
Gemini Evals
Configure and run Gemini for evals
GeminiModel
To authenticate with Gemini, you must pass either your credentials or a project, location pair. In the following example, we quickly instantiate the Gemini model as follows:
LiteLLM Evals
Configure and run LiteLLM for evals
Need to install the extra dependency litellm>=1.0.3
Here is an example of how to initialize LiteLLMModel for llama3 using ollama.
Groq
Groq provides ultra-low latency inference for LLMs through its custom-built LPU™ architecture.
Featured Tutorials
Google GenAI Tracing
Instrument LLM calls made using the Google Gen AI Python SDK
Launch Phoenix
Install
Setup
Use the register function to connect your application to Phoenix.
Observe
Now that you have tracing setup, all GenAI SDK requests will be streamed to Phoenix for observability and evaluation.
Anthropic
Anthropic is an AI research company that develops LLMs, including Claude, with a focus on alignment and reliable behavior.
Featured Tutorials
OpenAI Node.js SDK
Install
Setup
To load the OpenAI instrumentation, specify it in the registerInstrumentations call along with any additional instrumentation you wish to enable.
Support
Instrumentation version >1.0.0 supports both attribute masking and context attribute propagation to spans.
Resources
Anthropic Tracing
Install
Setup
Use the register function to connect your application to Phoenix:
Run Anthropic
A simple Anthropic application that is now instrumented
Observe
Now that you have tracing setup, all invocations of pipelines will be streamed to your running Phoenix for observability and evaluation.
Resources:
AutoGen
AutoGen is an open-source Python framework for orchestrating multi-agent LLM interactions with shared memory and tool integrations to build scalable AI workflows
Featured Tutorials
Anthropic Evals
Configure and run Anthropic for evals
AnthropicModel
Usage
In this section, we will showcase the methods and properties that our EvalModels have. First, instantiate your model from the. Once you've instantiated your model, you can get responses from the LLM by simply calling the model and passing a text string.
VertexAI Tracing
Instrument LLM calls made using VertexAI's SDK via the VertexAIInstrumentor
Launch Phoenix
Install
Setup
Use the register function to connect your application to Phoenix:
Run VertexAI
Observe
Now that you have tracing setup, all invocations of Vertex models will be streamed to your running Phoenix for observability and evaluation.
Resources
Agno
Agno is an open-source Python framework for building lightweight, model-agnostic AI agents with built-in memory, knowledge, tools, and reasoning capabilities
Hugging Face smolagents
Hugging Face smolagents is a minimalist Python library for building powerful AI agents with simple abstractions, tool integrations, and flexible LLM support
Featured Tutorials
LlamaIndex
LlamaIndex is an open-source framework that streamlines connecting, ingesting, indexing, and retrieving structured or unstructured data to power efficient, data-aware language model applications.
Featured Tutorials
Haystack
Haystack is an open-source framework for building scalable semantic search and QA pipelines with document indexing, retrieval, and reader components
Featured Tutorials
Guardrails AI
Guardrails is an open-source Python framework for adding programmable input/output validators to LLM applications, ensuring safe, structured, and compliant model interactions
Featured Tutorials
Portkey
Portkey is an AI Gateway and observability platform that provides routing, guardrails, caching, and monitoring for 200+ LLMs with enterprise-grade security and reliability features.
Prompt Flow
PromptFlow is a framework for designing, orchestrating, testing, and monitoring end-to-end LLM prompt workflows with built-in versioning and analytics
Vercel
Vercel is a cloud platform that simplifies building, deploying, and scaling modern web applications with features like serverless functions, edge caching, and seamless Git integration
Instructor
Instructor is a library that helps you define structured output formats for LLMs.
Website:
Don't see an integration you were looking for? We'd love to
E.g. , , , ,
E.g. , , ,
E.g. ,
Website:
Website:
Website:
Website:
Website:
Website:
Website:
Website:
You can choose among supported by LiteLLM. Make sure you have set the right environment variables set prior to initializing the model. For additional information about the environment variables for specific model providers visit:
Website:
Set the GEMINI_API_KEY environment variable. To use the GenAI SDK with Vertex AI instead of the Developer API, refer to Google's on setting the required environment variables.
This instrumentation will support tool calling soon. Refer to for the status.
Website:
This module provides automatic instrumentation for the . which may be used in conjunction with .
Anthropic is a leading provider for state-of-the-art LLMs. The Anthropic SDK can be instrumented using the package.
Website:
The VertexAI SDK can be instrumented using the package.
See Google's on setting up your environment for the Google Cloud AI Platform. You can also store your Project ID in the CLOUD_ML_PROJECT_ID environment variable.
from phoenix.otel import register
# configure the Phoenix tracer
tracer_provider = register(
project_name="my-llm-app", # Default is 'default'
auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)
class GeminiModel:
project: Optional[str] = None
location: Optional[str] = None
credentials: Optional["Credentials"] = None
model: str = "gemini-pro"
default_concurrency: int = 5
temperature: float = 0.0
max_tokens: int = 256
top_p: float = 1
top_k: int = 32
project = "my-project-id"
location = "us-central1" # as an example
model = GeminiModel(project=project, location=location)
model("Hello there, this is a tesst if you are working?")
# Output: "Hello world, I am working!"
class LiteLLMModel(BaseEvalModel):
model: str = "gpt-3.5-turbo"
"""The model name to use."""
temperature: float = 0.0
"""What sampling temperature to use."""
max_tokens: int = 256
"""The maximum number of tokens to generate in the completion."""
top_p: float = 1
"""Total probability mass of tokens to consider at each step."""
num_retries: int = 6
"""Maximum number to retry a model if an RateLimitError, OpenAIError, or
ServiceUnavailableError occurs."""
request_timeout: int = 60
"""Maximum number of seconds to wait when retrying."""
model_kwargs: Dict[str, Any] = field(default_factory=dict)
"""Model specific params"""
import os
from phoenix.evals import LiteLLMModel
os.environ["OLLAMA_API_BASE"] = "http://localhost:11434"
model = LiteLLMModel(model="ollama/llama3")
from phoenix.otel import register
# Configure the Phoenix tracer
tracer_provider = register(
project_name="my-llm-app", # Default is 'default'
auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)
import os
from google import genai
def send_message_multi_turn() -> tuple[str, str]:
client = genai.Client(api_key=os.environ["GEMINI_API_KEY"])
chat = client.chats.create(model="gemini-2.0-flash-001")
response1 = chat.send_message("What is the capital of France?")
response2 = chat.send_message("Why is the sky blue?")
return response1.text or "", response2.text or ""
from phoenix.otel import register
# configure the Phoenix tracer
tracer_provider = register(
project_name="my-llm-app", # Default is 'default'
auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)
class AnthropicModel(BaseModel):
model: str = "claude-2.1"
"""The model name to use."""
temperature: float = 0.0
"""What sampling temperature to use."""
max_tokens: int = 256
"""The maximum number of tokens to generate in the completion."""
top_p: float = 1
"""Total probability mass of tokens to consider at each step."""
top_k: int = 256
"""The cutoff where the model no longer selects the words."""
stop_sequences: List[str] = field(default_factory=list)
"""If the model encounters a stop sequence, it stops generating further tokens."""
extra_parameters: Dict[str, Any] = field(default_factory=dict)
"""Any extra parameters to add to the request body (e.g., countPenalty for a21 models)"""
max_content_size: Optional[int] = None
"""If you're using a fine-tuned model, set this to the maximum content size"""
model = #Instantiate your Anthropic model here
model("Hello there, how are you?")
# Output: "As an artificial intelligence, I don't have feelings,
# but I'm here and ready to assist you. How can I help you today?"
from phoenix.otel import register
# configure the Phoenix tracer
tracer_provider = register(
project_name="my-llm-app", # Default is 'default'
auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)
import vertexai
from vertexai.generative_models import GenerativeModel
vertexai.init(location="us-central1")
model = GenerativeModel("gemini-1.5-flash")
print(model.generate_content("Why is sky blue?").text)
Amazon Bedrock Evals
Configure and run Bedrock for evals
BedrockModel
class BedrockModel:
model_id: str = "anthropic.claude-v2"
"""The model name to use."""
temperature: float = 0.0
"""What sampling temperature to use."""
max_tokens: int = 256
"""The maximum number of tokens to generate in the completion."""
top_p: float = 1
"""Total probability mass of tokens to consider at each step."""
top_k: int = 256
"""The cutoff where the model no longer selects the words"""
stop_sequences: List[str] = field(default_factory=list)
"""If the model encounters a stop sequence, it stops generating further tokens. """
session: Any = None
"""A bedrock session. If provided, a new bedrock client will be created using this session."""
client = None
"""The bedrock session client. If unset, a new one is created with boto3."""
max_content_size: Optional[int] = None
"""If you're using a fine-tuned model, set this to the maximum content size"""
extra_parameters: Dict[str, Any] = field(default_factory=dict)
"""Any extra parameters to add to the request body (e.g., countPenalty for a21 models)"""
To Authenticate, the following code is used to instantiate a session and the session is used with Phoenix Evals
import boto3
# Create a Boto3 session
session = boto3.session.Session(
aws_access_key_id='ACCESS_KEY',
aws_secret_access_key='SECRET_KEY',
region_name='us-east-1' # change to your preferred AWS region
)
#If you need to assume a role
# Creating an STS client
sts_client = session.client('sts')
# (optional - if needed) Assuming a role
response = sts_client.assume_role(
RoleArn="arn:aws:iam::......",
RoleSessionName="AssumeRoleSession1",
#(optional) if MFA Required
SerialNumber='arn:aws:iam::...',
#Insert current token, needs to be run within x seconds of generation
TokenCode='PERIODIC_TOKEN'
)
# Your temporary credentials will be available in the response dictionary
temporary_credentials = response['Credentials']
# Creating a new Boto3 session with the temporary credentials
assumed_role_session = boto3.Session(
aws_access_key_id=temporary_credentials['AccessKeyId'],
aws_secret_access_key=temporary_credentials['SecretAccessKey'],
aws_session_token=temporary_credentials['SessionToken'],
region_name='us-east-1'
)
client_bedrock = assumed_role_session.client("bedrock-runtime")
# Arize Model Object - Bedrock ClaudV2 by default
model = BedrockModel(client=client_bedrock)
model("Hello there, how are you?")
# Output: "As an artificial intelligence, I don't have feelings,
# but I'm here and ready to assist you. How can I help you today?"
Add your OpenAI API key as an environment variable:
export OPENAI_API_KEY=[your_key_here]
Use the register function to connect your application to Phoenix:
from phoenix.otel import register
# configure the Phoenix tracer
tracer_provider = register(
project_name="my-llm-app", # Default is 'default'
auto_instrument=True # Auto-instrument your app based on installed dependencies
)
Now that you have tracing setup, all invocations of OpenAI (completions, chat completions, embeddings) will be streamed to your running Phoenix for observability and evaluation.
Connect to your Phoenix instance using the register function.
from phoenix.otel import register
# configure the Phoenix tracer
tracer_provider = register(
project_name="my-llm-app", # Default is 'default'
auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)
Run Groq
A simple Groq application that is now instrumented
import os
from groq import Groq
client = Groq(
# This is the default and can be omitted
api_key=os.environ.get("GROQ_API_KEY"),
)
chat_completion = client.chat.completions.create(
messages=[
{
"role": "user",
"content": "Explain the importance of low latency LLMs",
}
],
model="mixtral-8x7b-32768",
)
print(chat_completion.choices[0].message.content)
Observe
Now that you have tracing setup, all invocations of pipelines will be streamed to your running Phoenix for observability and evaluation.
Resources:
MistralAI Tracing
Instrument LLM calls made using MistralAI's SDK via the MistralAIInstrumentor
Set the MISTRAL_API_KEY environment variable to authenticate calls made using the SDK.
export MISTRAL_API_KEY=[your_key_here]
Connect to your Phoenix instance using the register function.
from phoenix.otel import register
# configure the Phoenix tracer
tracer_provider = register(
project_name="my-llm-app", # Default is 'default'
auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)
Run Mistral
import os
from mistralai import Mistral
from mistralai.models import UserMessage
api_key = os.environ["MISTRAL_API_KEY"]
model = "mistral-tiny"
client = Mistral(api_key=api_key)
chat_response = client.chat.complete(
model=model,
messages=[UserMessage(content="What is the best French cheese?")],
)
print(chat_response.choices[0].message.content)
Observe
Now that you have tracing setup, all invocations of Mistral (completions, chat completions, embeddings) will be streamed to your running Phoenix for observability and evaluation.
Resources
Flowise Tracing
Analyzing and troubleshooting what happens under the hood can be challenging without proper insights. By integrating your Flowise application with Phoenix, you can monitor traces and gain robust observability into your chatflows and agentflows.
Viewing Flowise traces in Phoenix
Access Configurations: Navigate to settings in your chatflow or agentflow and find configurations.
Connect to Phoenix: Go to the Analyze Chatflow tab and configure your application with Phoenix. Get your API key from your Phoenix instance to create your credentials. Be sure to name your project and confirm that the Phoenix toggle is enabled before saving.
View Traces: In Phoenix, you will find your project under the Projects tab. Click into this to view and analyze traces as you test your application.
Store and Experiment: Optionally, you can also filter traces, store traces in a dataset to run experiments, analyze patterns, and optimize your workflows over time.
LangFlow Tracing
Pull Langflow Repo
Navigate to the Langflow GitHub repo and pull the project down
Create .env file
Navigate to the repo and create a .env file with all the Arize Phoenix variables.
You can use the .env.example as a template to create the .env file
Add the following environment variable to the .env file
Start Docker Desktop, build the images, and run the container (this will take around 10 minutes the first time)
Go into your terminal into the Langflow directory and run the following commands
docker compose -f docker/dev.docker-compose.yml down || true
docker compose -f docker/dev.docker-compose.yml up --remove-orphans
Go to Hosted Langflow UI
Create a Flow
In this example, we'll use Simple Agent for this tutorial
Add your OpenAI Key to the Agent component in Langflow
Go into the Playground and run the Agent
Go to Arize Phoenix
Navigate to your project name (should match the name of of your Langflow Agent name)
Inspect Traces
AgentExecutor Trace is Arize Phoenix instrumentation to capture what's happening with the LangChain being ran during the Langflow components
The other UUID trace is the native Langflow tracing.
CrewAI
CrewAI is an open-source Python framework for orchestrating role-playing, autonomous AI agents into collaborative “crews” and “flows,” combining high-level simplicity with fine-grained control.
Featured Tutorials
LangChain
LangChain is an open-source framework for building language model applications with prompt chaining, memory, and external integrations
Featured Tutorials
Model Context Protocol
Anthropic's Model Context Protocol is a standard for connecting AI assistants to the systems where data lives, including content repositories, business tools, and development environments.
MistralAI
Mistral AI develops open-weight large language models, focusing on efficiency, customization, and cost-effective AI solutions.
LangChain.js
This module provides automatic instrumentation for LangChain.js, more specifically, the @langchain/core module. which may be used in conjunction with @opentelemetry/sdk-trace-node.
To load the LangChain instrumentation, manually instrument the @langchain/core/callbacks/manager module. The callbacks manager must be manually instrumented due to the non-traditional module structure in @langchain/core. Additional instrumentations can be registered as usual in the registerInstrumentations function.
import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node";
import {
LangChainInstrumentation
} from "@arizeai/openinference-instrumentation-langchain";
import * as CallbackManagerModule from "@langchain/core/callbacks/manager";
const provider = new NodeTracerProvider();
provider.register();
const lcInstrumentation = new LangChainInstrumentation();
// LangChain must be manually instrumented as it doesn't have
// a traditional module structure
lcInstrumentation.manuallyInstrument(CallbackManagerModule);
Support
Instrumentation version >1.0.0 supports both attribute masking and context attribute propagation to spans.
Instrumentation Version
LangChain ^0.3.0
LangChain ^0.2.0
LangChain ^0.1.0
>1.0.0
✅
✅
✅
>0.2.0
❌
✅
✅
>0.1.0
❌
❌
✅
Resources
LangGraph
LangGraph is an open-source framework for building graph-based LLM pipelines with modular nodes and seamless data integrations
Featured Tutorials
Pydantic AI
PydanticAI is a Python agent framework designed to make it less painful to build production-grade applications with Generative AI, built by the team behind Pydantic with type-safe structured outputs
DSPy
DSPy is an open-source Python framework for declaratively programming modular LLM pipelines and automatically optimizing prompts and model weights
Featured Tutorials
Qdrant
Qdrant is an open-source vector database built for high-dimensional vectors and large scale workflows
Phoenix can be used to trace and evaluate applications that use Qdrant as a vector database.
Examples
Pinecone
Pinecone is a vector database that can be used to power RAG in various applications.
Phoenix can be used to trace and evaluate applications that use Pinecone as a vector database.
Examples
MongoDB
MongoDB is a database platform. Their Atlas product is built for GenAI applications.
Website: mongodb.com
Phoenix can be used to trace and evaluate applications that use MongoDB Atlas as a vector database.
Featured Tutorials
Zilliz / Milvus
Milvus is an open-source vector database built for GenAI applications.
Phoenix can be used to trace and evaluate applications that use Zilliz or Milvus as a vector database.
Examples:
Amazon Bedrock Agents Tracing
Instrument LLM calls to AWS Bedrock via the boto3 client using the BedrockInstrumentor
Amazon Bedrock Agents allow you to easily define, deploy, and manage agents on your AWS infrastructure. Traces on invocations of these agents can be captured using OpenInference and viewed in Phoenix.
This instrumentation will capture data on LLM calls, action group invocations (as tools), knowledgebase lookups, and more.
Launch Phoenix
Install
Setup
Connect to your Phoenix instance using the register function.
After connecting to your Phoenix server, instrument boto3 prior to initializing a bedrock-runtime client. All clients created after instrumentation will send traces on all calls to invoke_model, invoke_agent, and their streaming variations.
Run Bedrock Agents
From here you can run Bedrock as normal
Observe
Now that you have tracing setup, all calls will be streamed to your running Phoenix for observability and evaluation.
Resources
OpenAI Evals
Configure and run OpenAI for evals
OpenAIModel
Need to install the extra dependencies openai>=1.0.0
All models newer than GPT 3.5 Turbo are tested regularly. If you're using an older model than that, you may run into deprecated API parameters.
To authenticate with OpenAI you will need, at a minimum, an API key. The model class will look for it in your environment, or you can pass it via argument as shown above. In addition, you can choose the specific name of the model you want to use and its configuration parameters. The default values specified above are common default values from OpenAI. Quickly instantiate your model as follows:
Azure OpenAI
The code snippet below shows how to initialize OpenAIModel for Azure:
Note that the model param is actually the engine of your deployment. You may get a DeploymentNotFound error if this parameter is not correct. You can find your engine param in the Azure OpenAI playground.
\
Azure OpenAI supports specific options:
Agno Tracing
Phoenix provides seamless observability and tracing for Agno agents through the OpenInference instrumentation package. This integration automatically captures agent interactions, tool usage, reasoning steps, and multi-agent conversations, giving you complete visibility into your Agno applications. Monitor performance, debug issues, and evaluate agent behavior in real-time as your agents execute complex workflows and collaborate in teams.
Agno is a lightweight, high-performance Python framework for building AI agents with tools, memory, and reasoning capabilities. It enables developers to create autonomous agents that can perform complex tasks, access knowledge bases, and collaborate in multi-agent teams. With support for 23+ model providers and lightning-fast performance (~3μs instantiation), Agno is designed for production-ready AI applications.
Key Features
Model Agnostic: Connect to OpenAI, Anthropic, Google, and 20+ other providers
Lightning Fast: Agents instantiate in ~3μs with minimal memory footprint
Built-in Reasoning: First-class support for chain-of-thought and reasoning models
Multi-Modal: Native support for text, image, audio, and video processing
Agentic RAG: Advanced retrieval-augmented generation with hybrid search
Multi-Agent Teams: Coordinate multiple agents for complex workflows
Production Ready: Pre-built FastAPI routes and monitoring capabilities
Launch Phoenix
Install
Setup
Use the register function to connect your application to Phoenix:
Run Agno
Observe
Now that you have tracing setup, all invocations of Agno agents will be streamed to Phoenix for observability and evaluation.
Resources
LlamaIndex Workflows Tracing
How to use the python LlamaIndexInstrumentor to trace LlamaIndex Workflows
Our LlamaIndexInstrumentor automatically captures traces for LlamaIndex Workflows agents. If you've already enabled that instrumentor, you do not need to complete the steps below.
We recommend using llama_index >= 0.11.0
Launch Phoenix
Install
Setup
Initialize the LlamaIndexInstrumentor before your application code. This instrumentor will trace both LlamaIndex Workflows calls, as well as calls to the general LlamaIndex package.
Run LlamaIndex Workflows
By instrumenting LlamaIndex, spans will be created whenever an agent is invoked and will be sent to the Phoenix server for collection.
Observe
Now that you have tracing setup, all invocations of chains will be streamed to your running Phoenix for observability and evaluation.
Resources
Weaviate
Weaviate is an open source, AI-native vector database.
Phoenix can be used to trace and evaluate applications that use Weaviate as a vector database.
Examples
Phoenix provides auto-instrumentation for the .
provides low latency and lightning-fast inference for AI models. Arize supports instrumenting Groq API calls, including role types such as system, user, and assistant messages, as well as tool use. You can create a free GroqCloud account and to get started.
MistralAI is a leading provider for state-of-the-art LLMs. The MistralAI SDK can be instrumented using the package.
Note: If you are using using an environment that is not , you may need to modify the Endpoint field.
You can also reference here.
Note: This Langflow integration is for
Website:
Website:
Website:
Website:
Website:
Website:
Website:
Website:
For full details on Azure OpenAI, check out the
are a subset of the LlamaIndex package specifically designed to support agent development.
Sign up for Phoenix:
Sign up for an Arize Phoenix account at
Install packages:
Set your Phoenix endpoint and API Key:
Your Phoenix API key can be found on the Keys section of your .
Launch your local Phoenix instance:
For details on customizing a local terminal deployment, see .
Install packages:
Set your Phoenix endpoint:
See Terminal for more details
Run your containerized instance:
This will expose the Phoenix on localhost:6006
Install packages:
Set your Phoenix endpoint:
Install packages:
Launch Phoenix:
Website:
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
from phoenix.otel import register
# configure the Phoenix tracer
tracer_provider = register(
project_name="my-llm-app", # Default is 'default'
auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)
class OpenAIModel:
api_key: Optional[str] = field(repr=False, default=None)
"""Your OpenAI key. If not provided, will be read from the environment variable"""
organization: Optional[str] = field(repr=False, default=None)
"""
The organization to use for the OpenAI API. If not provided, will default
to what's configured in OpenAI
"""
base_url: Optional[str] = field(repr=False, default=None)
"""
An optional base URL to use for the OpenAI API. If not provided, will default
to what's configured in OpenAI
"""
model: str = "gpt-4"
"""Model name to use. In of azure, this is the deployment name such as gpt-35-instant"""
temperature: float = 0.0
"""What sampling temperature to use."""
max_tokens: int = 256
"""The maximum number of tokens to generate in the completion.
-1 returns as many tokens as possible given the prompt and
the models maximal context size."""
top_p: float = 1
"""Total probability mass of tokens to consider at each step."""
frequency_penalty: float = 0
"""Penalizes repeated tokens according to frequency."""
presence_penalty: float = 0
"""Penalizes repeated tokens."""
n: int = 1
"""How many completions to generate for each prompt."""
model_kwargs: Dict[str, Any] = field(default_factory=dict)
"""Holds any model parameters valid for `create` call not explicitly specified."""
batch_size: int = 20
"""Batch size to use when passing multiple documents to generate."""
request_timeout: Optional[Union[float, Tuple[float, float]]] = None
"""Timeout for requests to OpenAI completion API. Default is 600 seconds."""
model = OpenAI()
model("Hello there, this is a test if you are working?")
# Output: "Hello! I'm working perfectly. How can I assist you today?"
model = OpenAIModel(
model="gpt-35-turbo-16k",
azure_endpoint="https://arize-internal-llm.openai.azure.com/",
api_version="2023-09-15-preview",
)
api_version: str = field(default=None)
"""
The verion of the API that is provisioned
https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#rest-api-versioning
"""
azure_endpoint: Optional[str] = field(default=None)
"""
The endpoint to use for azure openai. Available in the azure portal.
https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal#create-a-resource
"""
azure_deployment: Optional[str] = field(default=None)
azure_ad_token: Optional[str] = field(default=None)
azure_ad_token_provider: Optional[Callable[[], str]] = field(default=None)
from phoenix.otel import register
# configure the Phoenix tracer
tracer_provider = register(
project_name="my-llm-app", # Default is 'default'
auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.tools.duckduckgo import DuckDuckGoTools
agent = Agent(
model=OpenAIChat(id="gpt-4o-mini"),
tools=[DuckDuckGoTools()],
markdown=True,
debug_mode=True,
)
agent.run("What is currently trending on Twitter?")
docker pull arizephoenix/phoenix:latest
docker run -p 6006:6006 arizephoenix/phoenix:latest
pip install arize-phoenix-otel
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
AutoGen is an agent framework from Microsoft that allows for complex Agent creation. It is unique in its ability to create multiple agents that work together.
The AutoGen Agent framework allows creation of multiple agents and connection of those agents to work together to accomplish tasks.
Launch Phoenix
Sign up for Phoenix:
Install packages:
pip install arize-phoenix-otel
Set your Phoenix endpoint and API Key:
import os
# Add Phoenix API Key for tracing
PHOENIX_API_KEY = "ADD YOUR API KEY"
os.environ["PHOENIX_CLIENT_HEADERS"] = f"api_key={PHOENIX_API_KEY}"
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "https://app.phoenix.arize.com"
Launch your local Phoenix instance:
pip install arize-phoenix
phoenix serve
Install packages:
pip install arize-phoenix-otel
Set your Phoenix endpoint:
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
See Terminal for more details
docker pull arizephoenix/phoenix:latest
Run your containerized instance:
docker run -p 6006:6006 arizephoenix/phoenix:latest
This will expose the Phoenix on localhost:6006
Install packages:
pip install arize-phoenix-otel
Set your Phoenix endpoint:
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
Install packages:
pip install arize-phoenix
Launch Phoenix:
import phoenix as px
px.launch_app()
Install
Phoenix instruments Autogen by instrumenting the underlying model library it's using. If your agents are set up to call OpenAI, use our OpenAI instrumentor per the example below.
If your agents are using a different model, be sure to instrument that model instead by installing its respective OpenInference library.
Connect to your Phoenix instance using the register function.
from phoenix.otel import register
# configure the Phoenix tracer
tracer_provider = register(
project_name="my-llm-app", # Default is 'default'
auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)
Run Autogen
From here you can use Autogen as normal, and Phoenix will automatically trace any model calls made.
Observe
The Phoenix support is simple in its first incarnation but allows for capturing all of the prompt and responses that occur under the framework between each agent.
The individual prompt and responses are captured directly through OpenAI calls. If you're using a different underlying model provider than OpenAI, instrument your application using the respective instrumentor instead.
Resources:
Cleanlab
Ensuring the reliability and accuracy of LLM-generated responses is a critical challenge for production AI systems. Poor-quality training data, ambiguous labels, and untrustworthy outputs can degrade model performance and lead to unreliable results.
This guide demonstrates how to integrate Cleanlab’s Trustworthy Language Model (TLM) with Phoenix to systematically identify and improve low-quality LLM responses. By leveraging TLM for automated data quality assessment and Phoenix for response analysis, you can build more robust and trustworthy AI applications.
Specifically, this tutorial will walk through:
Evaluating LLM-generated responses for trustworthiness.
Using Cleanlab TLM to score and flag untrustworthy responses.
Leveraging Phoenix for tracing and visualizing response evaluations.
Key Implementation Steps for generating evals w/ TLM
Install Dependencies, Set up API Keys, Obtain LLM Reponses + Trace in Phoenix
Download Trace Dataset
import phoenix as px
spans_df = px.Client().get_spans_dataframe(project_name=[your_project_name])
spans_df.head()
Prep data from trace dataset
# Create a new DataFrame with input and output columns
eval_df = spans_df[["context.span_id", "attributes.input.value", "attributes.output.value"]].copy()
eval_df.set_index("context.span_id", inplace=True)
# Combine system and user prompts from the traces
def get_prompt(input_value):
if isinstance(input_value, str):
input_value = json.loads(input_value)
system_prompt = input_value["messages"][0]["content"]
user_prompt = input_value["messages"][1]["content"]
return system_prompt + "\n" + user_prompt
# Get the responses from the traces
def get_response(output_value):
if isinstance(output_value, str):
output_value = json.loads(output_value)
return output_value["choices"][0]["message"]["content"]
# Create a list of prompts and associated responses
prompts = [get_prompt(input_value) for input_value in eval_df["attributes.input.value"]]
responses = [get_response(output_value) for output_value in eval_df["attributes.output.value"]]
eval_df["prompt"] = prompts
eval_df["response"] = responses
Setup TLM & Evaluate each pair
from cleanlab_tlm import TLM
tlm = TLM(options={"log": ["explanation"]})
# Evaluate each of the prompt, response pairs using TLM
evaluations = tlm.get_trustworthiness_score(prompts, responses)
# Extract the trustworthiness scores and explanations from the evaluations
trust_scores = [entry["trustworthiness_score"] for entry in evaluations]
explanations = [entry["log"]["explanation"] for entry in evaluations]
# Add the trust scores and explanations to the DataFrame
eval_df["score"] = trust_scores
eval_df["explanation"] = explanations
Note: This package requires you to be using the Vercel AI SDK version 3.3 or higher.
Installation
npm install --save @arizeai/openinference-vercel
You will also need to install OpenTelemetry and Vercel packages to your project.
npm i @opentelemetry/api @vercel/otel @opentelemetry/exporter-trace-otlp-proto @arizeai/openinference-semantic-conventions
Usage
To process your Vercel AI SDK Spans add a OpenInferenceSimpleSpanProcessor or OpenInferenceBatchSpanProcessor to your OpenTelemetry configuration.
import { registerOTel } from "@vercel/otel";
import { diag, DiagConsoleLogger, DiagLogLevel } from "@opentelemetry/api";
import {
isOpenInferenceSpan,
OpenInferenceSimpleSpanProcessor,
} from "@arizeai/openinference-vercel";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-proto";
import { SEMRESATTRS_PROJECT_NAME } from "@arizeai/openinference-semantic-conventions";
// For troubleshooting, set the log level to DiagLogLevel.DEBUG
// This is not required and should not be added in a production setting
diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.DEBUG);
export function register() {
registerOTel({
serviceName: "phoenix-next-app",
attributes: {
// This is not required but it will allow you to send traces to a specific
// project in phoenix
[SEMRESATTRS_PROJECT_NAME]: "your-next-app",
},
spanProcessors: [
new OpenInferenceSimpleSpanProcessor({
exporter: new OTLPTraceExporter({
headers: {
// API key if you're sending it to Phoenix
api_key: process.env["PHOENIX_API_KEY"],
},
url:
process.env["PHOENIX_COLLECTOR_ENDPOINT"] ||
"https://app.phoenix.arize.com/v1/traces",
}),
spanFilter: (span) => {
// Only export spans that are OpenInference to remove non-generative spans
// This should be removed if you want to export all spans
return isOpenInferenceSpan(span);
},
}),
],
});
}
Now enable telemetry in your AI SDK calls by setting the experimental_telemetry parameter to true.
const result = await generateText({
model: openai("gpt-4-turbo"),
prompt: "Write a short story about a cat.",
experimental_telemetry: { isEnabled: true },
});
Examples
Prompt Flow Tracing
Create flows using Microsoft PromptFlow and send their traces to Phoenix
Launch Phoenix
Install
pip install promptflow
Setup
Set up the OpenTelemetry endpoint to point to Phoenix and use Prompt flow's setup_exporter_from_environ to start tracing any further flows and LLM calls.
import os
from opentelemetry.sdk.environment_variables import OTEL_EXPORTER_OTLP_ENDPOINT
from promptflow.tracing._start_trace import setup_exporter_from_environ
endpoint = f"{os.environ["PHOENIX_COLLECTOR_ENDPOINT]}/v1/traces" # replace with your Phoenix endpoint if self-hosting
os.environ[OTEL_EXPORTER_OTLP_ENDPOINT] = endpoint
setup_exporter_from_environ()
Run PromptFlow
Observe
You should see the spans render in Phoenix as shown in the below screenshots.
To instrument your application, import and enable BeeAIInstrumentation
Create the instrumentation.js file:
// instrumentation.js
import { NodeSDK, node, resources } from "@opentelemetry/sdk-node";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-proto";
import { ATTR_SERVICE_NAME } from "@opentelemetry/semantic-conventions";
import { SEMRESATTRS_PROJECT_NAME } from "@arizeai/openinference-semantic-conventions";
import { BeeAIInstrumentation } from "@arizeai/openinference-instrumentation-beeai";
import * as beeaiFramework from "beeai-framework";
// Initialize Instrumentation Manually
const beeAIInstrumentation = new BeeAIInstrumentation();
const provider = new NodeSDK({
resource: new resources.Resource({
[ATTR_SERVICE_NAME]: "beeai",
[SEMRESATTRS_PROJECT_NAME]: "beeai-project",
}),
spanProcessors: [
new node.SimpleSpanProcessor(
new OTLPTraceExporter({
url: "http://localhost:6006/v1/traces",
}),
),
],
instrumentations: [beeAIInstrumentation],
});
await provider.start();
// Manually Patch BeeAgent (This is needed when the module is not loaded via require (commonjs))
console.log("🔧 Manually instrumenting BeeAgent...");
beeAIInstrumentation.manuallyInstrument(beeaiFramework);
console.log("✅ BeeAgent manually instrumented.");
// eslint-disable-next-line no-console
console.log("👀 OpenInference initialized");
Import the library and call the BeeAI framework
import "./instrumentation";
import { BeeAgent } from "beeai-framework/agents/bee/agent";
import { TokenMemory } from "beeai-framework/memory/tokenMemory";
import { DuckDuckGoSearchTool } from "beeai-framework/tools/search/duckDuckGoSearch";
import { OpenMeteoTool } from "beeai-framework/tools/weather/openMeteo";
import { OllamaChatModel } from "beeai-framework/adapters/ollama/backend/chat";
const llm = new OllamaChatModel("llama3.1");
const agent = new BeeAgent({
llm,
memory: new TokenMemory(),
tools: [new DuckDuckGoSearchTool(), new OpenMeteoTool()],
});
const response = await agent.run({
prompt: "What's the current weather in Berlin?",
});
console.log(`Agent 🤖 : `, response.result.text);
LiteLLM Tracing
Any calls made to the following functions will be automatically captured by this integration:
completion()
acompletion()
completion_with_retries()
embedding()
aembedding()
image_generation()
aimage_generation()
Launch Phoenix
Install
Setup
Use the register function to connect your application to Phoenix:
Add any API keys needed by the models you are using with LiteLLM.
Run LiteLLM
You can now use LiteLLM as normal and calls will be traces in Phoenix.
Observe
Traces should now be visible in Phoenix!
Resources
OpenAI Agents SDK Tracing
Use Phoenix and OpenAI Agents SDK for powerful multi-agent tracing
1. Launch Phoenix
We have several code samples below on different ways to integrate with OpenAI, based on how you want to use Phoenix.
2. Install
3. Set up Tracing
Add your OpenAI API key as an environment variable:
Use the register function to connect your application to Phoenix:
Run your agentscode.
View your traces in Phoenix.
Resources
Guardrails AI Tracing
Instrument LLM applications that use the Guardrails AI framework
Launch Phoenix
Sign up for Phoenix:
Install packages:
Set your Phoenix endpoint and API Key:
Launch your local Phoenix instance:
Install packages:
Set your Phoenix endpoint:
See Terminal for more details
Run your containerized instance:
This will expose the Phoenix on localhost:6006
Install packages:
Set your Phoenix endpoint:
Install packages:
Launch Phoenix:
Install
Setup
Connect to your Phoenix instance using the register function.
Run Guardrails
From here, you can run Guardrails as normal:
Observe
Now that you have tracing setup, all invocations of underlying models used by Guardrails (completions, chat completions, embeddings) will be streamed to your running Phoenix for observability and evaluation. Additionally, Guards will be present as a new span kind in Phoenix.
Resources
LangChain Tracing
How to use the python LangChainInstrumentor to trace LangChain
Launch Phoenix
Install
Setup
Use the register function to connect your application to Phoenix:
Run LangChain
By instrumenting LangChain, spans will be created whenever a chain is run and will be sent to the Phoenix server for collection.
Observe
Now that you have tracing setup, all invocations of chains will be streamed to your running Phoenix for observability and evaluation.
Resources
LangGraph Tracing
LangGraph is supported by our LangChain instrumentor. If you've already set up instrumentation with LangChain, you don't need to complete the set up below
Launch Phoenix
Install
Install the OpenInference Langchain library before your application code. Our LangChainInstrumentor works for both standard LangChain applications and for LangGraph agents.
Setup
Use the register function to connect your application to Phoenix:
Run LangGraph
By instrumenting LangGraph, spans will be created whenever an agent is invoked and will be sent to the Phoenix server for collection.
Observe
Now that you have tracing setup, all invocations of chains will be streamed to your running Phoenix for observability and evaluation.
Resources
Instructor Tracing
Launch Phoenix
Install
Be sure you also install the OpenInference library for the underlying model you're using along with Instructor. For example, if you're using OpenAI calls directly, you would also add: openinference-instrumentation-openai
Setup
Connect to your Phoenix instance using the register function.
Run Instructor
From here you can use instructor as normal.
Observe
Now that you have tracing setup, all invocations of your underlying model (completions, chat completions, embeddings) and instructor triggers will be streamed to your running Phoenix for observability and evaluation.
Ragas
This guide will walk you through the process of creating and evaluating agents using Ragas and Arize Phoenix. We'll cover the following steps:
Build a customer support agent with the OpenAI Agents SDK
Trace agent activity to monitor interactions
Generate a benchmark dataset for performance analysis
Evaluate agent performance using Ragas
We will walk through the key steps in the documentation below. Check out the full tutorial here:
Creating the Agent
Here we've setup a basic agent that can solve math problems. We have a function tool that can solve math equations, and an agent that can use this tool. We'll use the Runner class to run the agent and get the final output.
Evaluating the Agent
Agents can go awry for a variety of reasons. We can use Ragas to evaluate whether the agent responded correctly. Two Ragas measurements help with this:
Tool Call Accuracy - Did our agent choose the right tool with the right arguments?
Agent Goal Accuracy - Did our agent accomplish the stated goal and get to the right outcome?
We'll import both metrics we're measuring from Ragas, and use the multi_turn_ascore(sample) to get the results. The AgentGoalAccuracyWithReference metric compares the final output to the reference to see if the goal was accomplished. The ToolCallAccuracy metric compares the tool call to the reference tool call to see if the tool call was made correctly.
In the notebook, we also define the helper function conversation_to_ragas_sample which converts the agent messages into a format that Ragas can use.
The following code snippets define our task function and evaluators.
Run the Experiment
Once we've generated a dataset of questions, we can use our experiments feature to track changes across models, prompts, parameters for the agent.
Finally, we run our experiment and view the results in Phoenix.
Haystack Tracing
Instrument LLM applications built with Haystack
Launch Phoenix
Sign up for Phoenix:
Install packages:
Set your Phoenix endpoint and API Key:
Launch your local Phoenix instance:
Install packages:
Set your Phoenix endpoint:
See Terminal for more details
Run your containerized instance:
This will expose the Phoenix on localhost:6006
Install packages:
Set your Phoenix endpoint:
Install packages:
Launch Phoenix:
Install
Setup
Use the register function to connect your application to Phoenix:
Run Haystack
Phoenix's auto-instrumentor collects any traces from Haystack Pipelines. If you are using Haystack but not using Pipelines, you won't see traces appearing in Phoenix automatically.
If you don't want to use Haystack pipelines but still want tracing in Phoenix, you can use instead of this auto-instrumentor.
From here, you can set up your Haystack app as normal:
Observe
Now that you have tracing setup, all invocations of pipelines will be streamed to your running Phoenix for observability and evaluation.
Instrumenting and Evaluating a Weaviate RAG Pipeline
This tutorial serves as a great starting point to see how to manually instrument a RAG chatbot built on Weaviate, and visualize and evaluate the results in Phoenix.
Sign up for an Arize Phoenix account at
Your Phoenix API key can be found on the Keys section of your .
For details on customizing a local terminal deployment, see .
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
is a tool that estimates the trustworthiness of an LLM response. It provides a confidence score that helps detect hallucinations, ambiguous responses, and potential misinterpretations. This enables teams to flag unreliable outputs and improve the robustness of their AI systems.
This package provides a set of utilities to ingest (>= 3.3) spans into platforms like and .
@arizeai/openinference-vercel provides a set of utilities to help you ingest Vercel AI SDK spans into platforms and works in conjunction with Vercel's OpenTelemetry support. To get started, you will need to add OpenTelemetry support to your Vercel project according to their .
Note: The OpenInferenceSpanProcessor does not handle the exporting of spans so you will need to pass it an as a parameter.
For details on Vercel AI SDK telemetry see the .
To see an example go to the in the .
For more information on Vercel OpenTelemetry support see the .
This integration will allow you to trace flows and send their traces into.
Sign up for Phoenix:
Sign up for an Arize Phoenix account at
Install packages:
pipinstallarize-phoenix-otel
Set your Phoenix endpoint and API Key:
Your Phoenix API key can be found on the Keys section of your .
Launch your local Phoenix instance:
pipinstallarize-phoenixphoenixserve
For details on customizing a local terminal deployment, see .
Install packages:
pipinstallarize-phoenix-otel
Set your Phoenix endpoint:
See Terminal for more details
docker pull arizephoenix/phoenix:latest
Run your containerized instance:
docker run -p 6006:6006 arizephoenix/phoenix:latest
This will expose the Phoenix on localhost:6006
Install packages:
pip install arize-phoenix-otel
Set your Phoenix endpoint:
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
Install packages:
pip install arize-phoenix
Launch Phoenix:
import phoenix as px
px.launch_app()
Proceed with creating Prompt flow flows as usual. See this for inspiration.
BeeAI has native integration with Arize Phoenix
NPM Version
This module provides automatic instrumentation for . It integrates seamlessly with the package to collect and export telemetry data.
For more information on OpenTelemetry Node.js SDK, see the .
allows developers to call all LLM APIs using the openAI format. is a proxy server to call 100+ LLMs in OpenAI format. Both are supported by this auto-instrumentation.
Sign up for Phoenix:
Sign up for an Arize Phoenix account at
Install packages:
Set your Phoenix endpoint and API Key:
Your Phoenix API key can be found on the Keys section of your .
Launch your local Phoenix instance:
For details on customizing a local terminal deployment, see .
Install packages:
Set your Phoenix endpoint:
See Terminal for more details
Run your containerized instance:
This will expose the Phoenix on localhost:6006
Install packages:
Set your Phoenix endpoint:
Install packages:
Launch Phoenix:
Sign up for Phoenix:
Sign up for an Arize Phoenix account at
Install packages:
Set your Phoenix endpoint and API Key:
Your Phoenix API key can be found on the Keys section of your .
Launch your local Phoenix instance:
For details on customizing a local terminal deployment, see .
Install packages:
Set your Phoenix endpoint:
See Terminal for more details
Run your containerized instance:
This will expose the Phoenix on localhost:6006
Install packages:
Set your Phoenix endpoint:
Install packages:
Launch Phoenix:
In this example we will instrument a small program that uses the framework to protect their LLM calls.
Sign up for an Arize Phoenix account at
Your Phoenix API key can be found on the Keys section of your .
For details on customizing a local terminal deployment, see .
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
Phoenix has first-class support for applications.
Sign up for Phoenix:
Sign up for an Arize Phoenix account at
Install packages:
Set your Phoenix endpoint and API Key:
Your Phoenix API key can be found on the Keys section of your .
Launch your local Phoenix instance:
For details on customizing a local terminal deployment, see .
Install packages:
Set your Phoenix endpoint:
See Terminal for more details
Run your containerized instance:
This will expose the Phoenix on localhost:6006
Install packages:
Set your Phoenix endpoint:
Install packages:
Launch Phoenix:
Phoenix has first-class support for applications.
Sign up for Phoenix:
Sign up for an Arize Phoenix account at
Install packages:
Set your Phoenix endpoint and API Key:
Your Phoenix API key can be found on the Keys section of your .
Launch your local Phoenix instance:
For details on customizing a local terminal deployment, see .
Install packages:
Set your Phoenix endpoint:
See Terminal for more details
Run your containerized instance:
This will expose the Phoenix on localhost:6006
Install packages:
Set your Phoenix endpoint:
Install packages:
Launch Phoenix:
Sign up for Phoenix:
Sign up for an Arize Phoenix account at
Install packages:
Set your Phoenix endpoint and API Key:
Your Phoenix API key can be found on the Keys section of your .
Launch your local Phoenix instance:
For details on customizing a local terminal deployment, see .
Install packages:
Set your Phoenix endpoint:
See Terminal for more details
Run your containerized instance:
This will expose the Phoenix on localhost:6006
Install packages:
Set your Phoenix endpoint:
Install packages:
Launch Phoenix:
is a library that provides robust evaluation metrics for LLM applications, making it easy to assess quality. When integrated with Phoenix, it enriches your experiments with metrics like goal accuracy and tool call accuracy—helping you evaluate performance more effectively and track improvements over time.
Phoenix provides auto-instrumentation for
Sign up for an Arize Phoenix account at
Your Phoenix API key can be found on the Keys section of your .
For details on customizing a local terminal deployment, see .
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
from phoenix.otel import register
# configure the Phoenix tracer
tracer_provider = register(
project_name="my-llm-app", # Default is 'default'
auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)
import os
os.environ["OPENAI_API_KEY"] = "PASTE_YOUR_API_KEY_HERE"
import litellm
completion_response = litellm.completion(model="gpt-3.5-turbo",
messages=[{"content": "What's the capital of China?", "role": "user"}])
print(completion_response)
docker pull arizephoenix/phoenix:latest
docker run -p 6006:6006 arizephoenix/phoenix:latest
pip install arize-phoenix-otel
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
from phoenix.otel import register
# configure the Phoenix tracer
tracer_provider = register(
project_name="agents", # Default is 'default'
auto_instrument=True # Auto-instrument your app based on installed dependencies
)
from agents import Agent, Runner
agent = Agent(name="Assistant", instructions="You are a helpful assistant")
result = Runner.run_sync(agent, "Write a haiku about recursion in programming.")
print(result.final_output)
pip install arize-phoenix-otel
import os
# Add Phoenix API Key for tracing
PHOENIX_API_KEY = "ADD YOUR API KEY"
os.environ["PHOENIX_CLIENT_HEADERS"] = f"api_key={PHOENIX_API_KEY}"
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "https://app.phoenix.arize.com"
pip install arize-phoenix
phoenix serve
pip install arize-phoenix-otel
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
docker pull arizephoenix/phoenix:latest
docker run -p 6006:6006 arizephoenix/phoenix:latest
pip install arize-phoenix-otel
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
from phoenix.otel import register
# configure the Phoenix tracer
tracer_provider = register(
project_name="my-llm-app", # Default is 'default'
auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)
from guardrails import Guard
from guardrails.hub import TwoWords
import openai
guard = Guard().use(
TwoWords(),
)
response = guard(
llm_api=openai.chat.completions.create,
prompt="What is another name for America?",
model="gpt-3.5-turbo",
max_tokens=1024,
)
print(response)
docker pull arizephoenix/phoenix:latest
docker run -p 6006:6006 arizephoenix/phoenix:latest
pip install arize-phoenix-otel
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
from phoenix.otel import register
# configure the Phoenix tracer
tracer_provider = register(
project_name="my-llm-app", # Default is 'default'
auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)
from phoenix.otel import register
# configure the Phoenix tracer
tracer_provider = register(
project_name="my-llm-app", # Default is 'default'
auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)
docker pull arizephoenix/phoenix:latest
docker run -p 6006:6006 arizephoenix/phoenix:latest
pip install arize-phoenix-otel
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
from phoenix.otel import register
# configure the Phoenix tracer
tracer_provider = register(
project_name="my-llm-app", # Default is 'default'
auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)
import instructor
from pydantic import BaseModel
from openai import OpenAI
# Define your desired output structure
class UserInfo(BaseModel):
name: str
age: int
# Patch the OpenAI client
client = instructor.from_openai(OpenAI())
# Extract structured data from natural language
user_info = client.chat.completions.create(
model="gpt-3.5-turbo",
response_model=UserInfo,
messages=[{"role": "user", "content": "John Doe is 30 years old."}],
)
print(user_info.name)
#> John Doe
print(user_info.age)
#> 30
from agents import Runner, function_tool
@function_tool
def solve_equation(equation: str) -> str:
"""Use python to evaluate the math equation, instead of thinking about it yourself.
Args:"
equation: string which to pass into eval() in python
"""
return str(eval(equation))
from agents import Agent
agent = Agent(
name="Math Solver",
instructions="You solve math problems by evaluating them with python and returning the result",
tools=[solve_equation],
)
import asyncio
from agents import Runner
async def solve_math_problem(input):
if isinstance(input, dict):
input = next(iter(input.values()))
result = await Runner.run(agent, input)
return {"final_output": result.final_output, "messages": result.to_input_list()}
import os
# Add Phoenix API Key for tracing
PHOENIX_API_KEY = "ADD YOUR API KEY"
os.environ["PHOENIX_CLIENT_HEADERS"] = f"api_key={PHOENIX_API_KEY}"
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "https://app.phoenix.arize.com"
pip install arize-phoenix
phoenix serve
pip install arize-phoenix-otel
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
docker pull arizephoenix/phoenix:latest
docker run -p 6006:6006 arizephoenix/phoenix:latest
pip install arize-phoenix-otel
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
from phoenix.otel import register
# configure the Phoenix tracer
tracer_provider = register(
project_name="my-llm-app", # Default is 'default'
auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)
from haystack import Pipeline
from haystack.components.generators import OpenAIGenerator
from haystack.components.builders.prompt_builder import PromptBuilder
prompt_template = """
Answer the following question.
Question: {{question}}
Answer:
"""
# Initialize the pipeline
pipeline = Pipeline()
# Initialize the OpenAI generator component
llm = OpenAIGenerator(model="gpt-3.5-turbo")
prompt_builder = PromptBuilder(template=prompt_template)
# Add the generator component to the pipeline
pipeline.add_component("prompt_builder", prompt_builder)
pipeline.add_component("llm", llm)
pipeline.connect("prompt_builder", "llm")
# Define the question
question = "What is the location of the Hanging Gardens of Babylon?"
Instructor tracing
MCP Tracing
Trace your MCP Clients and Servers in a unified view
import os# Add Phoenix API Key for tracingPHOENIX_API_KEY ="ADD YOUR API KEY"os.environ["PHOENIX_CLIENT_HEADERS"]=f"api_key={PHOENIX_API_KEY}"os.environ["PHOENIX_COLLECTOR_ENDPOINT"]="https://app.phoenix.arize.com"
import os# Add Phoenix API Key for tracingPHOENIX_API_KEY ="ADD YOUR API KEY"os.environ["PHOENIX_CLIENT_HEADERS"]=f"api_key={PHOENIX_API_KEY}"os.environ["PHOENIX_COLLECTOR_ENDPOINT"]="https://app.phoenix.arize.com"
import os# Add Phoenix API Key for tracingPHOENIX_API_KEY ="ADD YOUR API KEY"os.environ["PHOENIX_CLIENT_HEADERS"]=f"api_key={PHOENIX_API_KEY}"os.environ["PHOENIX_COLLECTOR_ENDPOINT"]="https://app.phoenix.arize.com"
import os# Add Phoenix API Key for tracingPHOENIX_API_KEY ="ADD YOUR API KEY"os.environ["PHOENIX_CLIENT_HEADERS"]=f"api_key={PHOENIX_API_KEY}"os.environ["PHOENIX_COLLECTOR_ENDPOINT"]="https://app.phoenix.arize.com"
import os# Add Phoenix API Key for tracingPHOENIX_API_KEY ="ADD YOUR API KEY"os.environ["PHOENIX_CLIENT_HEADERS"]=f"api_key={PHOENIX_API_KEY}"os.environ["PHOENIX_COLLECTOR_ENDPOINT"]="https://app.phoenix.arize.com"
import os# Add Phoenix API Key for tracingPHOENIX_API_KEY ="ADD YOUR API KEY"os.environ["PHOENIX_CLIENT_HEADERS"]=f"api_key={PHOENIX_API_KEY}"os.environ["PHOENIX_COLLECTOR_ENDPOINT"]="https://app.phoenix.arize.com"
import os# Add Phoenix API Key for tracingPHOENIX_API_KEY ="ADD YOUR API KEY"os.environ["PHOENIX_CLIENT_HEADERS"]=f"api_key={PHOENIX_API_KEY}"os.environ["PHOENIX_COLLECTOR_ENDPOINT"]="https://app.phoenix.arize.com"
DSPy uses LiteLLM under the hood to make some calls. By adding the OpenInference library for LiteLLM, you'll be able to see additional information like token counts on your traces.
Setup
Connect to your Phoenix instance using the register function.
from phoenix.otel import register
# configure the Phoenix tracer
tracer_provider = register(
project_name="my-llm-app", # Default is 'default'
auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)
Run DSPy
Now run invoke your compiled DSPy module. Your traces should appear inside of Phoenix.
class BasicQA(dspy.Signature):
"""Answer questions with short factoid answers."""
question = dspy.InputField()
answer = dspy.OutputField(desc="often between 1 and 5 words")
if __name__ == "__main__":
turbo = dspy.OpenAI(model="gpt-3.5-turbo")
dspy.settings.configure(lm=turbo)
with using_attributes(
session_id="my-test-session",
user_id="my-test-user",
metadata={
"test-int": 1,
"test-str": "string",
"test-list": [1, 2, 3],
"test-dict": {
"key-1": "val-1",
"key-2": "val-2",
},
},
tags=["tag-1", "tag-2"],
prompt_template_version="v1.0",
prompt_template_variables={
"city": "Johannesburg",
"date": "July 11th",
},
):
# Define the predictor.
generate_answer = dspy.Predict(BasicQA)
# Call the predictor on a particular input.
pred = generate_answer(
question="What is the capital of the united states?" # noqa: E501
) # noqa: E501
print(f"Predicted Answer: {pred.answer}")
Observe
Now that you have tracing setup, all predictions will be streamed to your running Phoenix for observability and evaluation.
Resources
smolagents Tracing
How to use the SmolagentsInstrumentor to trace smolagents by Hugging Face
smolagents is a minimalist AI agent framework developed by Hugging Face, designed to simplify the creation and deployment of powerful agents with just a few lines of code. It focuses on simplicity and efficiency, making it easy for developers to leverage large language models (LLMs) for various applications.
Phoenix provides auto-instrumentation, allowing you to track and visualize every step and call made by your agent.
Launch Phoenix
We have several code samples below on different ways to integrate with smolagents, based on how you want to use Phoenix.
Sign up for Phoenix:
Install packages:
pip install arize-phoenix-otel
Set your Phoenix endpoint and API Key:
import os
# Add Phoenix API Key for tracing
PHOENIX_API_KEY = "ADD YOUR API KEY"
os.environ["PHOENIX_CLIENT_HEADERS"] = f"api_key={PHOENIX_API_KEY}"
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "https://app.phoenix.arize.com"
Launch your local Phoenix instance:
pip install arize-phoenix
phoenix serve
Install packages:
pip install arize-phoenix-otel
Set your Phoenix endpoint:
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
See Terminal for more details
docker pull arizephoenix/phoenix:latest
Run your containerized instance:
docker run -p 6006:6006 arizephoenix/phoenix:latest
This will expose the Phoenix on localhost:6006
Install packages:
pip install arize-phoenix-otel
Set your Phoenix endpoint:
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
Connect to your Phoenix instance using the register function.
from phoenix.otel import register
# configure the Phoenix tracer
tracer_provider = register(
project_name="my-llm-app", # Default is 'default'
auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)
Create & Run an Agent
Create your Hugging Face Model, and at every run, traces will be sent to Phoenix.
from smolagents import (
CodeAgent,
ToolCallingAgent,
ManagedAgent,
DuckDuckGoSearchTool,
VisitWebpageTool,
HfApiModel,
)
model = HfApiModel()
agent = ToolCallingAgent(
tools=[DuckDuckGoSearchTool(), VisitWebpageTool()],
model=model,
)
managed_agent = ManagedAgent(
agent=agent,
name="managed_agent",
description="This is an agent that can do web search.",
)
manager_agent = CodeAgent(
tools=[],
model=model,
managed_agents=[managed_agent],
)
manager_agent.run(
"If the US keeps its 2024 growth rate, how many years will it take for the GDP to double?"
)
Observe
Now that you have tracing setup, all invocations and steps of your Agent will be streamed to your running Phoenix for observability and evaluation.
Resources
Portkey Tracing
How to trace Portkey AI Gateway requests with Phoenix for comprehensive LLM observability
Use the register function to connect your application to Phoenix:
from phoenix.otel import register
# configure the Phoenix tracer
tracer_provider = register(
project_name="my-portkey-app", # Default is 'default'
auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)
Run Portkey
By instrumenting Portkey, spans will be created whenever requests are made through the AI Gateway and will be sent to the Phoenix server for collection.
Basic Usage with OpenAI
import os
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders
# Set up your API keys
os.environ["OPENAI_API_KEY"] = "your-openai-api-key"
os.environ["PORTKEY_API_KEY"] = "your-portkey-api-key" # Optional for self-hosted
client = OpenAI(
api_key=os.environ.get("OPENAI_API_KEY"),
base_url=PORTKEY_GATEWAY_URL,
default_headers=createHeaders(
provider="openai",
api_key=os.environ.get("PORTKEY_API_KEY")
)
)
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "What is artificial intelligence?"}]
)
print(response.choices[0].message.content)
Using Portkey SDK Directly
from portkey_ai import Portkey
# Initialize Portkey client
portkey = Portkey(
api_key="your-portkey-api-key", # Optional for self-hosted
virtual_key="your-openai-virtual-key" # Or use provider-specific virtual keys
)
response = portkey.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Explain machine learning"}]
)
print(response.choices[0].message.content)
Observe
Now that you have tracing setup, all requests through Portkey's AI Gateway will be streamed to your running Phoenix instance for observability and evaluation. You'll be able to see:
Request/Response Traces: Complete visibility into LLM interactions
Routing Decisions: Which provider was selected and why
Fallback Events: When and why fallbacks were triggered
Cache Performance: Hit/miss rates and response times
Cost Tracking: Token usage and costs across providers
Latency Metrics: Response times for each provider and route
Resources
Mastra Tracing
Launch Phoenix
Setup
Install packages:
npm install @arizeai/openinference-mastra
Initialize OpenTelemetry tracing for your Mastra application:
import { Mastra } from "@mastra/core";
import {
OpenInferenceOTLPTraceExporter,
isOpenInferenceSpan,
} from "@arizeai/openinference-mastra";
export const mastra = new Mastra({
// ... other config
telemetry: {
serviceName: "openinference-mastra-agent", // you can rename this to whatever you want to appear in the Phoenix UI
enabled: true,
export: {
type: "custom",
exporter: new OpenInferenceOTLPTraceExporter({
url: process.env.PHOENIX_COLLECTOR_ENDPOINT,
headers: {
Authorization: `Bearer ${process.env.PHOENIX_API_KEY}`, // if you're self-hosting Phoenix without auth, you can remove this header
},
// optional: filter out http, and other node service specific spans
// they will still be exported to Mastra, but not to the target of
// this exporter
spanFilter: isOpenInferenceSpan,
}),
},
},
});
From here you can use Mastra as normal. All agents, workflows, and tool calls will be automatically traced.
Example Agent Walkthrough
Here is a full project example to get you started:
Launch Phoenix using one of the methods above
The rest of this tutorial will assume you are running Phoenix locally on the default localhost:6006 port.
Create a new Mastra project
npm create mastra@latest
# answer the prompts, include agent, tools, and the example when asked
cd chosen-project-name
npm install --save @arizeai/openinference-mastra
Connect to Phoenix
Add the OpenInference telemetry code to your index.js file. The complete file should now look like this:
Model Details: Model name, provider, token usage, response metadata
Performance Metrics: Execution time, token counts, costs
User Context: Session IDs, user information (if provided)
You can view all of this information in the Phoenix UI to debug issues, optimize performance, and understand your application's behavior.
Phoenix MCP Server
Phoenix MCP Server is an implementation of the Model Context Protocol for the Arize Phoenix platform. It provides a unified interface to Phoenix's capabilites.
Phoenix MCP Server supports:
Prompts Management: Create, list, update, and iterate on prompts
Datasets: Explore datasets, and synthesize new examples
Experiments: Pull experiment results and visualize them with the help of an LLM
Connecting the Phoenix MCP Server
Using the Phoenix MCP Server
The MCP server can be used to interact with Prompts, Experiments, and Datasets. It can be used to retrieve information about each item, and can create and update Prompts.
Some good questions to try:
What prompts do I have in Phoenix?
Create a new prompt in Phoenix that classifies user intent
Update my classification prompt in Phoenix with these new options
Summarize the Phoenix experiments run on my agent inputs dataset
Visualize the results of my jailbreak dataset experiments in Phoenix
Hoping to see additional functionality?
CrewAI Tracing
Instrument multi agent applications using CrewAI
Launch Phoenix
Sign up for Phoenix:
Install packages:
Set your Phoenix endpoint and API Key:
Launch your local Phoenix instance:
Install packages:
Set your Phoenix endpoint:
See Terminal for more details
Run your containerized instance:
This will expose the Phoenix on localhost:6006
Install packages:
Set your Phoenix endpoint:
Install packages:
Launch Phoenix:
Install
CrewAI uses either Langchain or LiteLLM under the hood to call models, depending on the version.
If you're using CrewAI<0.63.0, we recommend installing our openinference-instrumentation-langchain library to get visibility of LLM calls.
If you're using CrewAI>= 0.63.0, we recommend instead adding our openinference-instrumentation-litellm library to get visibility of LLM calls.
Setup
Connect to your Phoenix instance using the register function.
Run CrewAI
From here, you can run CrewAI as normal
Observe
Now that you have tracing setup, all calls to your Crew will be streamed to your running Phoenix for observability and evaluation.
Resources
MCP Tracing
Phoenix provides tracing for MCP clients and servers through OpenInference. This includes the unique capability to trace client to server interactions under a single trace in the correct hierarchy.
The openinference-instrumentation-mcp instrumentor is unique compared to other OpenInference instrumentors. It does not generate any of its own telemetry. Instead, it enables context propagation between MCP clients and servers to unify traces. You still need generate OpenTelemetry traces in both the client and server to see a unified trace.
Install
Because the MCP instrumentor does not generate its own telemetry, you must use it alongside other instrumentation code to see traces.
The example code below uses OpenAI agents, which you can instrument using:
Add Tracing to your MCP Client
Add Tracing to your MCP Server
Observe
Now that you have tracing setup, all invocations of your client and server will be streamed to Phoenix for observability and evaluation, and connected in the platform.
is a framework for automatically prompting and fine-tuning language models. It provides composable and declarative APIs that allow developers to describe the architecture of their LLM application in the form of a "module" (inspired by PyTorch's nn.Module). It them compiles these modules using "teleprompters" that optimize the module for a particular task. The term "teleprompter" is meant to evoke "prompting at a distance," and could involve selecting few-shot examples, generating prompts, or fine-tuning language models.
Sign up for Phoenix:
Sign up for an Arize Phoenix account at
Install packages:
pipinstallarize-phoenix-otel
Set your Phoenix endpoint and API Key:
import os# Add Phoenix API Key for tracingPHOENIX_API_KEY ="ADD YOUR API KEY"os.environ["PHOENIX_CLIENT_HEADERS"]=f"api_key={PHOENIX_API_KEY}"os.environ["PHOENIX_COLLECTOR_ENDPOINT"]="https://app.phoenix.arize.com"
Your Phoenix API key can be found on the Keys section of your .
Launch your local Phoenix instance:
pipinstallarize-phoenixphoenixserve
For details on customizing a local terminal deployment, see .
docker run -p 6006:6006 arizephoenix/phoenix:latest
This will expose the Phoenix on localhost:6006
Install packages:
pip install arize-phoenix-otel
Set your Phoenix endpoint:
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
Install packages:
pip install arize-phoenix
Launch Phoenix:
import phoenix as px
px.launch_app()
Traces and spans from an instrumented DSPy custom module.
Sign up for an Arize Phoenix account at
Your Phoenix API key can be found on the Keys section of your .
For details on customizing a local terminal deployment, see .
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
Phoenix provides seamless integration with , the AI Gateway and observability platform that routes to 200+ LLMs with enterprise-grade features including guardrails, caching, and load balancing.
Sign up for Phoenix:
Sign up for an Arize Phoenix account at
Install packages:
pipinstallarize-phoenix-otel
Set your Phoenix endpoint and API Key:
import os# Add Phoenix API Key for tracingPHOENIX_API_KEY ="ADD YOUR API KEY"os.environ["PHOENIX_CLIENT_HEADERS"]=f"api_key={PHOENIX_API_KEY}"os.environ["PHOENIX_COLLECTOR_ENDPOINT"]="https://app.phoenix.arize.com"
Your Phoenix API key can be found on the Keys section of your .
Launch your local Phoenix instance:
pipinstallarize-phoenixphoenixserve
For details on customizing a local terminal deployment, see .
Your Phoenix API key can be found on the Keys section of your .
For details on customizing a local terminal deployment, see .
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
import os
# Add Phoenix API Key for tracing
PHOENIX_API_KEY = "ADD YOUR API KEY"
os.environ["PHOENIX_CLIENT_HEADERS"] = f"api_key={PHOENIX_API_KEY}"
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "https://app.phoenix.arize.com"
pip install arize-phoenix
phoenix serve
pip install arize-phoenix-otel
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
docker pull arizephoenix/phoenix:latest
docker run -p 6006:6006 arizephoenix/phoenix:latest
pip install arize-phoenix-otel
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
from phoenix.otel import register
# configure the Phoenix tracer
tracer_provider = register(
project_name="my-llm-app", # Default is 'default'
auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)
import os
from crewai import Agent, Task, Crew, Process
from crewai_tools import SerperDevTool
os.environ["OPENAI_API_KEY"] = "YOUR_OPENAI_API_KEY"
os.environ["SERPER_API_KEY"] = "YOUR_SERPER_API_KEY"
search_tool = SerperDevTool()
# Define your agents with roles and goals
researcher = Agent(
role='Senior Research Analyst',
goal='Uncover cutting-edge developments in AI and data science',
backstory="""You work at a leading tech think tank.
Your expertise lies in identifying emerging trends.
You have a knack for dissecting complex data and presenting actionable insights.""",
verbose=True,
allow_delegation=False,
# You can pass an optional llm attribute specifying what model you wanna use.
# llm=ChatOpenAI(model_name="gpt-3.5", temperature=0.7),
tools=[search_tool]
)
writer = Agent(
role='Tech Content Strategist',
goal='Craft compelling content on tech advancements',
backstory="""You are a renowned Content Strategist, known for your insightful and engaging articles.
You transform complex concepts into compelling narratives.""",
verbose=True,
allow_delegation=True
)
# Create tasks for your agents
task1 = Task(
description="""Conduct a comprehensive analysis of the latest advancements in AI in 2024.
Identify key trends, breakthrough technologies, and potential industry impacts.""",
expected_output="Full analysis report in bullet points",
agent=researcher
)
task2 = Task(
description="""Using the insights provided, develop an engaging blog
post that highlights the most significant AI advancements.
Your post should be informative yet accessible, catering to a tech-savvy audience.
Make it sound cool, avoid complex words so it doesn't sound like AI.""",
expected_output="Full blog post of at least 4 paragraphs",
agent=writer
)
# Instantiate your crew with a sequential process
crew = Crew(
agents=[researcher, writer],
tasks=[task1, task2],
verbose=2, # You can set it to 1 or 2 to different logging levels
process = Process.sequential
)
# Get your crew to work!
result = crew.kickoff()
print("######################")
print(result)
import asyncio
from agents import Agent, Runner
from agents.mcp import MCPServer, MCPServerStdio
from dotenv import load_dotenv
from phoenix.otel import register
load_dotenv()
# Connect to your Phoenix instance
tracer_provider = register(auto_instrument=True)
async def run(mcp_server: MCPServer):
agent = Agent(
name="Assistant",
instructions="Use the tools to answer the users question.",
mcp_servers=[mcp_server],
)
while True:
message = input("\n\nEnter your question (or 'exit' to quit): ")
if message.lower() == "exit" or message.lower() == "q":
break
print(f"\n\nRunning: {message}")
result = await Runner.run(starting_agent=agent, input=message)
print(result.final_output)
async def main():
async with MCPServerStdio(
name="Financial Analysis Server",
params={
"command": "fastmcp",
"args": ["run", "./server.py"],
},
client_session_timeout_seconds=30,
) as server:
await run(server)
if __name__ == "__main__":
asyncio.run(main())
import json
import os
from datetime import datetime, timedelta
import openai
from dotenv import load_dotenv
from mcp.server.fastmcp import FastMCP
from pydantic import BaseModel
from phoenix.otel import register
load_dotenv()
# You must also connect your MCP server to Phoenix
tracer_provider = register(auto_instrument=True)
# Get a tracer to add additional instrumentattion
tracer = tracer_provider.get_tracer("financial-analysis-server")
# Configure OpenAI client
client = openai.OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))
MODEL = "gpt-4-turbo"
# Create MCP server
mcp = FastMCP("Financial Analysis Server")
class StockAnalysisRequest(BaseModel):
ticker: str
time_period: str = "short-term" # short-term, medium-term, long-term
@mcp.tool()
@tracer.tool(name="MCP.analyze_stock") # this OpenInference call adds tracing to this method
def analyze_stock(request: StockAnalysisRequest) -> dict:
"""Analyzes a stock based on its ticker symbol and provides investment recommendations."""
# Make LLM API call to analyze the stock
prompt = f"""
Provide a detailed financial analysis for the stock ticker: {request.ticker}
Time horizon: {request.time_period}
Please include:
1. Company overview
2. Recent financial performance
3. Key metrics (P/E ratio, market cap, etc.)
4. Risk assessment
5. Investment recommendation
Format your response as a JSON object with the following structure:
{{
"ticker": "{request.ticker}",
"company_name": "Full company name",
"overview": "Brief company description",
"financial_performance": "Analysis of recent performance",
"key_metrics": {{
"market_cap": "Value in billions",
"pe_ratio": "Current P/E ratio",
"dividend_yield": "Current yield percentage",
"52_week_high": "Value",
"52_week_low": "Value"
}},
"risk_assessment": "Analysis of risks",
"recommendation": "Buy/Hold/Sell recommendation with explanation",
"time_horizon": "{request.time_period}"
}}
"""
response = client.chat.completions.create(
model=MODEL,
messages=[{"role": "user", "content": prompt}],
response_format={"type": "json_object"},
)
analysis = json.loads(response.choices[0].message.content)
return analysis
# ... define any additional MCP tools you wish
if __name__ == "__main__":
mcp.run()
This tutorial will show you how to embed a large volume of data, upload it to a vector database, run top K similarity searches against it, and monitor it in production using VectorFlow, Arize Phoenix, Weaviate and LlamaIndex.
How to use the python LlamaIndexInstrumentor to trace LlamaIndex
For LlamaIndex, tracing instrumentation is added via an OpenTelemetry instrumentor aptly named the LlamaIndexInstrumentor . This callback is what is used to create spans and send them to the Phoenix collector.
Launch Phoenix
Sign up for Phoenix:
Phoenix Developer Edition is another name for LlamaTrace
Install packages:
pip install arize-phoenix-otel
Set your Phoenix endpoint and API Key:
import os
# Add Phoenix API Key for tracing
PHOENIX_API_KEY = "ADD YOUR API KEY"
os.environ["PHOENIX_CLIENT_HEADERS"] = f"api_key={PHOENIX_API_KEY}"
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "https://app.phoenix.arize.com"
Launch your local Phoenix instance:
pip install arize-phoenix
phoenix serve
Install packages:
pip install arize-phoenix-otel
Set your Phoenix endpoint:
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
See Terminal for more details
docker pull arizephoenix/phoenix:latest
Run your containerized instance:
docker run -p 6006:6006 arizephoenix/phoenix:latest
This will expose the Phoenix on localhost:6006
Install packages:
pip install arize-phoenix-otel
Set your Phoenix endpoint:
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
Install packages:
pip install arize-phoenix
Launch Phoenix:
import phoenix as px
px.launch_app()
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See self-hosting or use one of the other deployment options to retain traces.
Initialize the LlamaIndexInstrumentor before your application code.
from openinference.instrumentation.llama_index import LlamaIndexInstrumentor
from phoenix.otel import register
tracer_provider = register()
LlamaIndexInstrumentor().instrument(tracer_provider=tracer_provider)
Run LlamaIndex
You can now use LlamaIndex as normal, and tracing will be automatically captured and sent to your Phoenix instance.
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
import os
os.environ["OPENAI_API_KEY"] = "YOUR OPENAI API KEY"
documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
response = query_engine.query("Some question about the data should go here")
print(response)
Observe
View your traces in Phoenix:
Resources
Legacy Integrations (<0.10.43)
Legacy One-Click (<0.10.43)
Using phoenix as a callback requires an install of `llama-index-callbacks-arize-phoenix>0.1.3'
llama-index 0.10 introduced modular sub-packages. To use llama-index's one click, you must install the small integration first:
# Phoenix can display in real time the traces automatically
# collected from your LlamaIndex application.
import phoenix as px
# Look for a URL in the output to open the App in a browser.
px.launch_app()
# The App is initially empty, but as you proceed with the steps below,
# traces will appear automatically as your LlamaIndex application runs.
from llama_index.core import set_global_handler
set_global_handler("arize_phoenix")
# Run all of your LlamaIndex applications as usual and traces
# will be collected and displayed in Phoenix.
Legacy (<0.10.0)
If you are using an older version of llamaIndex (pre-0.10), you can still use phoenix. You will have to be using arize-phoenix>3.0.0 and downgrade openinference-instrumentation-llama-index<1.0.0
# Phoenix can display in real time the traces automatically
# collected from your LlamaIndex application.
import phoenix as px
# Look for a URL in the output to open the App in a browser.
px.launch_app()
# The App is initially empty, but as you proceed with the steps below,
# traces will appear automatically as your LlamaIndex application runs.
import llama_index
llama_index.set_global_handler("arize_phoenix")
# Run all of your LlamaIndex applications as usual and traces
# will be collected and displayed in Phoenix.
Pydantic AI Tracing
How to use the python PydanticAIInstrumentor to trace PydanticAI agents
Launch Phoenix
Install
Setup
Set up tracing using OpenTelemetry and the PydanticAI instrumentation:
Basic Usage
Here's a simple example using PydanticAI with automatic tracing:
Advanced Usage
Agent with System Prompts and Tools
Observe
Now that you have tracing setup, all PydanticAI agent operations will be streamed to your running Phoenix instance for observability and evaluation. You'll be able to see:
Agent interactions: Complete conversations between your application and the AI model
Structured outputs: Pydantic model validation and parsing results
Tool usage: When agents call external tools and their responses
Performance metrics: Response times, token usage, and success rates
Error handling: Validation errors, API failures, and retry attempts
Multi-agent workflows: Complex interactions between multiple agents
The traces will provide detailed insights into your AI agent behaviors, making it easier to debug issues, optimize performance, and ensure reliability in production.
is a data framework for your LLM application. It's a powerful framework by which you can build an application that leverages RAG (retrieval-augmented generation) to super-charge an LLM with your own data. RAG is an extremely powerful LLM application model because it lets you harness the power of LLMs such as OpenAI's GPT but tuned to your data and use-case.
Phoenix supports LlamaIndex's latest paradigm. This paradigm requires LlamaIndex >= 0.10.43. For legacy support, see below.
Sign up for an Arize Phoenix account at or
Your Phoenix API key can be found on the Keys section of your .
For details on customizing a local terminal deployment, see .
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
is a Python agent framework designed to make it less painful to build production-grade applications with Generative AI. Built by the team behind Pydantic, it provides a clean, type-safe way to build AI agents with structured outputs.
Sign up for Phoenix:
Sign up for an Arize Phoenix account at
Install packages:
Set your Phoenix endpoint and API Key:
Your Phoenix API key can be found on the Keys section of your .
Launch your local Phoenix instance:
For details on customizing a local terminal deployment, see .
Install packages:
Set your Phoenix endpoint:
See Terminal for more details
Run your containerized instance:
This will expose the Phoenix on localhost:6006
Install packages:
Set your Phoenix endpoint:
Install packages:
Launch Phoenix:
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
import os
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace import TracerProvider
from openinference.instrumentation.pydantic_ai import OpenInferenceSpanProcessor
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
# Set up the tracer provider
tracer_provider = TracerProvider()
trace.set_tracer_provider(tracer_provider)
# Add the OpenInference span processor
endpoint = f"{os.environ['PHOENIX_COLLECTOR_ENDPOINT']}/v1/traces"
exporter = OTLPSpanExporter(endpoint=endpoint)
tracer_provider.add_span_processor(OpenInferenceSpanProcessor())
tracer_provider.add_span_processor(SimpleSpanProcessor(exporter))
from pydantic import BaseModel
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIModel
import nest_asyncio
nest_asyncio.apply()
# Set your OpenAI API key
os.environ["OPENAI_API_KEY"] = "YOUR_OPENAI_API_KEY"
# Define your Pydantic model
class LocationModel(BaseModel):
city: str
country: str
# Create and configure the agent
model = OpenAIModel("gpt-4", provider='openai')
agent = Agent(model, output_type=LocationModel, instrument=True)
# Run the agent
result = agent.run_sync("The windy city in the US of A.")
print(result)
from pydantic import BaseModel, Field
from pydantic_ai import Agent, RunContext
from pydantic_ai.models.openai import OpenAIModel
from typing import List
import httpx
class WeatherInfo(BaseModel):
location: str
temperature: float = Field(description="Temperature in Celsius")
condition: str
humidity: int = Field(description="Humidity percentage")
# Create an agent with system prompts and tools
weather_agent = Agent(
model=OpenAIModel("gpt-4"),
output_type=WeatherInfo,
system_prompt="You are a helpful weather assistant. Always provide accurate weather information.",
instrument=True
)
@weather_agent.tool
async def get_weather_data(ctx: RunContext[None], location: str) -> str:
"""Get current weather data for a location."""
# Mock weather API call - replace with actual weather service
async with httpx.AsyncClient() as client:
# This is a placeholder - use a real weather API
mock_data = {
"temperature": 22.5,
"condition": "partly cloudy",
"humidity": 65
}
return f"Weather in {location}: {mock_data}"
# Run the agent with tool usage
result = weather_agent.run_sync("What's the weather like in Paris?")
print(result)
import os# Add Phoenix API Key for tracingPHOENIX_API_KEY ="ADD YOUR API KEY"os.environ["PHOENIX_CLIENT_HEADERS"]=f"api_key={PHOENIX_API_KEY}"os.environ["PHOENIX_COLLECTOR_ENDPOINT"]="https://app.phoenix.arize.com"
How to use Pydantic Evals with Phoenix to evaluate AI applications using structured evaluation frameworks
Launch Phoenix
Install
Setup
Enable Phoenix tracing to capture traces from your application:
Basic Usage
1. Generate Traces to Evaluate
First, create some example traces by running your AI application. Here's a simple example:
2. Export Traces from Phoenix
Export the traces you want to evaluate:
3. Define Evaluation Dataset
Create a dataset of test cases using Pydantic Evals:
4. Create Custom Evaluators
Define evaluators to assess your model's performance:
5. Setup Task and Dataset
Create a task that retrieves outputs from your traced data:
6. Add LLM Judge Evaluator
For more sophisticated evaluation, add an LLM judge:
7. Run Evaluation
Execute the evaluation:
Advanced Usage
Upload Results to Phoenix
Upload your evaluation results back to Phoenix for visualization:
Custom Evaluation Workflows
You can create more complex evaluation workflows by combining multiple evaluators:
Observe
Once you have evaluation results uploaded to Phoenix, you can:
View evaluation metrics: See overall performance across different evaluation criteria
Analyze individual cases: Drill down into specific examples that passed or failed
Compare evaluators: Understand how different evaluation methods perform
Track improvements: Monitor evaluation scores over time as you improve your application
Debug failures: Identify patterns in failed evaluations to guide improvements
The Phoenix UI will display your evaluation results with detailed breakdowns, making it easy to understand your AI application's performance and identify areas for improvement.
import os# Add Phoenix API Key for tracingPHOENIX_API_KEY ="ADD YOUR API KEY"os.environ["PHOENIX_CLIENT_HEADERS"]=f"api_key={PHOENIX_API_KEY}"os.environ["PHOENIX_COLLECTOR_ENDPOINT"]="https://app.phoenix.arize.com"
Your Phoenix API key can be found on the Keys section of your .
Launch your local Phoenix instance:
pipinstallarize-phoenixphoenixserve
For details on customizing a local terminal deployment, see .
docker run -p 6006:6006 arizephoenix/phoenix:latest
This will expose the Phoenix on localhost:6006
Install packages:
pip install arize-phoenix-otel
Set your Phoenix endpoint:
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
Install packages:
pip install arize-phoenix
Launch Phoenix:
import phoenix as px
px.launch_app()
Sign up for Phoenix:
Sign up for an Arize Phoenix account at
Install packages:
pipinstallarize-phoenix-otel
Set your Phoenix endpoint and API Key:
import os# Add Phoenix API Key for tracingPHOENIX_API_KEY ="ADD YOUR API KEY"os.environ["PHOENIX_CLIENT_HEADERS"]=f"api_key={PHOENIX_API_KEY}"os.environ["PHOENIX_COLLECTOR_ENDPOINT"]="https://app.phoenix.arize.com"
Your Phoenix API key can be found on the Keys section of your .
Launch your local Phoenix instance:
pipinstallarize-phoenixphoenixserve
For details on customizing a local terminal deployment, see .
docker run -p 6006:6006 arizephoenix/phoenix:latest
This will expose the Phoenix on localhost:6006
Install packages:
pip install arize-phoenix-otel
Set your Phoenix endpoint:
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
Install packages:
pip install arize-phoenix
Launch Phoenix:
import phoenix as px
px.launch_app()
Sign up for Phoenix:
Sign up for an Arize Phoenix account at
Install packages:
pipinstallarize-phoenix-otel
Set your Phoenix endpoint and API Key:
import os# Add Phoenix API Key for tracingPHOENIX_API_KEY ="ADD YOUR API KEY"os.environ["PHOENIX_CLIENT_HEADERS"]=f"api_key={PHOENIX_API_KEY}"os.environ["PHOENIX_COLLECTOR_ENDPOINT"]="https://app.phoenix.arize.com"
Your Phoenix API key can be found on the Keys section of your .
Launch your local Phoenix instance:
pipinstallarize-phoenixphoenixserve
For details on customizing a local terminal deployment, see .
docker run -p 6006:6006 arizephoenix/phoenix:latest
This will expose the Phoenix on localhost:6006
Install packages:
pip install arize-phoenix-otel
Set your Phoenix endpoint:
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
Install packages:
pip install arize-phoenix
Launch Phoenix:
import phoenix as px
px.launch_app()
Sign up for Phoenix:
Sign up for an Arize Phoenix account at
Install packages:
pipinstallarize-phoenix-otel
Set your Phoenix endpoint and API Key:
import os# Add Phoenix API Key for tracingPHOENIX_API_KEY ="ADD YOUR API KEY"os.environ["PHOENIX_CLIENT_HEADERS"]=f"api_key={PHOENIX_API_KEY}"os.environ["PHOENIX_COLLECTOR_ENDPOINT"]="https://app.phoenix.arize.com"
Your Phoenix API key can be found on the Keys section of your .
Launch your local Phoenix instance:
pipinstallarize-phoenixphoenixserve
For details on customizing a local terminal deployment, see .
docker run -p 6006:6006 arizephoenix/phoenix:latest
This will expose the Phoenix on localhost:6006
Install packages:
pip install arize-phoenix-otel
Set your Phoenix endpoint:
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
Install packages:
pip install arize-phoenix
Launch Phoenix:
import phoenix as px
px.launch_app()
Sign up for Phoenix:
Sign up for an Arize Phoenix account at
Install packages:
pipinstallarize-phoenix-otel
Set your Phoenix endpoint and API Key:
import os# Add Phoenix API Key for tracingPHOENIX_API_KEY ="ADD YOUR API KEY"os.environ["PHOENIX_CLIENT_HEADERS"]=f"api_key={PHOENIX_API_KEY}"os.environ["PHOENIX_COLLECTOR_ENDPOINT"]="https://app.phoenix.arize.com"
Your Phoenix API key can be found on the Keys section of your .
Launch your local Phoenix instance:
pipinstallarize-phoenixphoenixserve
For details on customizing a local terminal deployment, see .
docker run -p 6006:6006 arizephoenix/phoenix:latest
This will expose the Phoenix on localhost:6006
Install packages:
pip install arize-phoenix-otel
Set your Phoenix endpoint:
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
Install packages:
pip install arize-phoenix
Launch Phoenix:
import phoenix as px
px.launch_app()
Sign up for Phoenix:
Sign up for an Arize Phoenix account at
Install packages:
pipinstallarize-phoenix-otel
Set your Phoenix endpoint and API Key:
import os# Add Phoenix API Key for tracingPHOENIX_API_KEY ="ADD YOUR API KEY"os.environ["PHOENIX_CLIENT_HEADERS"]=f"api_key={PHOENIX_API_KEY}"os.environ["PHOENIX_COLLECTOR_ENDPOINT"]="https://app.phoenix.arize.com"
Your Phoenix API key can be found on the Keys section of your .
Launch your local Phoenix instance:
pipinstallarize-phoenixphoenixserve
For details on customizing a local terminal deployment, see .
docker run -p 6006:6006 arizephoenix/phoenix:latest
This will expose the Phoenix on localhost:6006
Install packages:
pip install arize-phoenix-otel
Set your Phoenix endpoint:
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
Install packages:
pip install arize-phoenix
Launch Phoenix:
import phoenix as px
px.launch_app()
Sign up for Phoenix:
Sign up for an Arize Phoenix account at
Install packages:
pipinstallarize-phoenix-otel
Set your Phoenix endpoint and API Key:
import os# Add Phoenix API Key for tracingPHOENIX_API_KEY ="ADD YOUR API KEY"os.environ["PHOENIX_CLIENT_HEADERS"]=f"api_key={PHOENIX_API_KEY}"os.environ["PHOENIX_COLLECTOR_ENDPOINT"]="https://app.phoenix.arize.com"
Your Phoenix API key can be found on the Keys section of your .
Launch your local Phoenix instance:
pipinstallarize-phoenixphoenixserve
For details on customizing a local terminal deployment, see .
import os# Add Phoenix API Key for tracingPHOENIX_API_KEY ="ADD YOUR API KEY"os.environ["PHOENIX_CLIENT_HEADERS"]=f"api_key={PHOENIX_API_KEY}"os.environ["PHOENIX_COLLECTOR_ENDPOINT"]="https://app.phoenix.arize.com"
Your Phoenix API key can be found on the Keys section of your .
Launch your local Phoenix instance:
pipinstallarize-phoenixphoenixserve
For details on customizing a local terminal deployment, see .
docker run -p 6006:6006 arizephoenix/phoenix:latest
This will expose the Phoenix on localhost:6006
Install packages:
pip install arize-phoenix-otel
Set your Phoenix endpoint:
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
Install packages:
pip install arize-phoenix
Launch Phoenix:
import phoenix as px
px.launch_app()
is an evaluation library that provides preset direct evaluations and LLM Judge evaluations. It can be used to run evaluations over dataframes of cases defined with Pydantic models. This guide shows you how to use Pydantic Evals alongside Arize Phoenix to run evaluations on traces captured from your running application.
Sign up for Phoenix:
Sign up for an Arize Phoenix account at
Install packages:
Set your Phoenix endpoint and API Key:
Your Phoenix API key can be found on the Keys section of your .
Launch your local Phoenix instance:
For details on customizing a local terminal deployment, see .
Install packages:
Set your Phoenix endpoint:
See Terminal for more details
Run your containerized instance:
This will expose the Phoenix on localhost:6006
Install packages:
Set your Phoenix endpoint:
Install packages:
Launch Phoenix:
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
from openai import OpenAI
import os
client = OpenAI()
inputs = [
"What is the capital of France?",
"Who wrote Romeo and Juliet?",
"What is the largest planet in our solar system?",
]
def generate_trace(input):
client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{
"role": "system",
"content": "You are a helpful assistant. Only respond with the answer to the question as a single word or proper noun.",
},
{"role": "user", "content": input},
],
)
for input in inputs:
generate_trace(input)
from pydantic_evals import Case, Dataset
cases = [
Case(
name="capital of France",
inputs="What is the capital of France?",
expected_output="Paris"
),
Case(
name="author of Romeo and Juliet",
inputs="Who wrote Romeo and Juliet?",
expected_output="William Shakespeare",
),
Case(
name="largest planet",
inputs="What is the largest planet in our solar system?",
expected_output="Jupiter",
),
]
from pydantic_evals.evaluators import Evaluator, EvaluatorContext
class MatchesExpectedOutput(Evaluator[str, str]):
def evaluate(self, ctx: EvaluatorContext[str, str]) -> float:
is_correct = ctx.expected_output == ctx.output
return is_correct
class FuzzyMatchesOutput(Evaluator[str, str]):
def evaluate(self, ctx: EvaluatorContext[str, str]) -> float:
from difflib import SequenceMatcher
def similarity_ratio(a, b):
return SequenceMatcher(None, a, b).ratio()
# Consider it correct if similarity is above 0.8 (80%)
is_correct = similarity_ratio(ctx.expected_output, ctx.output) > 0.8
return is_correct
from pydantic_evals.evaluators import LLMJudge
dataset.add_evaluator(
LLMJudge(
rubric="Output and Expected Output should represent the same answer, even if the text doesn't match exactly",
include_input=True,
model="openai:gpt-4o-mini",
),
)
import os# Add Phoenix API Key for tracingPHOENIX_API_KEY ="ADD YOUR API KEY"os.environ["PHOENIX_CLIENT_HEADERS"]=f"api_key={PHOENIX_API_KEY}"os.environ["PHOENIX_COLLECTOR_ENDPOINT"]="https://app.phoenix.arize.com"
Your Phoenix API key can be found on the Keys section of your .
Launch your local Phoenix instance:
pipinstallarize-phoenixphoenixserve
This will expose the Phoenix on localhost:6006
For details on customizing a local terminal deployment, see .
Set your Phoenix endpoint and API Key:
export PHOENIX_COLLECTOR_ENDPOINT="http://localhost:6006/v1/traces"export PHOENIX_API_KEY="YOUR PHOENIX API KEY"# only necessary if you've enabled auth
Pull latest Phoenix image from:
dockerpullarizephoenix/phoenix:latest
Run your containerized instance:
dockerrun-p6006:6006arizephoenix/phoenix:latest
This will expose the Phoenix on localhost:6006
Set your Phoenix endpoint and API Key:
export PHOENIX_COLLECTOR_ENDPOINT="http://localhost:6006/v1/traces"export PHOENIX_API_KEY="YOUR PHOENIX API KEY"# only necessary if you've enabled auth
For more info on using Phoenix with Docker, see .
Sign up for Phoenix:
Sign up for an Arize Phoenix account at
Install packages:
Set your Phoenix endpoint and API Key:
Your Phoenix API key can be found on the Keys section of your .
Launch your local Phoenix instance:
For details on customizing a local terminal deployment, see .
Install packages:
Set your Phoenix endpoint:
See for more details
Run your containerized instance:
This will expose the Phoenix on localhost:6006
Install packages:
Set your Phoenix endpoint:
Install packages:
Launch Phoenix:
Pull latest Phoenix image from:
For more info on using Phoenix with Docker, see .
By default, notebook instances do not have persistent storage, so your traces will disappear after the notebook is closed. See or use one of the other deployment options to retain traces.
docker run -p 6006:6006 arizephoenix/phoenix:latest
pip install arize-phoenix-otel
import os
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = "http://localhost:6006"
pip install arize-phoenix
import phoenix as px
px.launch_app()
Terminal
pipinstallarize-phoenix-otel
import os# Add Phoenix API Key for tracingPHOENIX_API_KEY ="ADD YOUR API KEY"os.environ["PHOENIX_CLIENT_HEADERS"]=f"api_key={PHOENIX_API_KEY}"os.environ["PHOENIX_COLLECTOR_ENDPOINT"]="https://app.phoenix.arize.com"
