Cost Tracking

Phoenix allows you to track token-based costs for LLM runs automatically. The costs are calculated from token counts and model pricing data, then rolled up to the trace and project level for comprehensive cost analysis.

In most cases it is simplest to let Phoenix handle cost calculation using its built-in model pricing table. When custom pricing is required, you can create custom cost configurations in Settings > Models.

Send token counts

For Phoenix to accurately derive costs for LLM spans, you need to provide token counts in your traces:

• If you are using OpenInference auto-instrumentation with OpenAI, Anthropic, or other supported instrumentation, token counts and model information are automatically captured.

• If you are manually instrumenting your code, you should include the appropriate token count attributes in your spans.

• If you are using OpenTelemetry directly, ensure that your LLM spans include the OpenInference semantic conventions for token counts.

Required span attributes for cost tracking

Phoenix uses the OpenInference semantic conventions for cost tracking. The following attributes are required:

Basic token counts

Attribute	Type	Description
llm.token_count.prompt	Integer	The number of tokens in the prompt
llm.token_count.completion	Integer	The number of tokens in the completion
llm.token_count.total	Integer	Total number of tokens, including prompt and completion

Model information

Attribute	Type	Description
llm.model_name	String	The name of the language model being utilized
llm.provider	String	The hosting provider of the llm (e.g., openai, anthropic, azure)

Detailed token breakdown (optional)

For more granular cost tracking, you can provide detailed token counts:

Prompt token details

Attribute	Type	Description
llm.token_count.prompt_details.cache_read	Integer	The number of tokens read from previously cached prompts
llm.token_count.prompt_details.cache_write	Integer	The number of tokens written to cache
llm.token_count.prompt_details.audio	Integer	The number of audio input tokens presented in the prompt

Completion token details

Attribute	Type	Description
llm.token_count.completion_details.reasoning	Integer	The number of tokens used for model reasoning
llm.token_count.completion_details.audio	Integer	The number of audio input tokens generated by the model

Model pricing configuration

Phoenix includes a comprehensive model pricing table with built-in support for popular models from:

• OpenAI: GPT-3.5, GPT-4, GPT-4 Turbo, GPT-4o, and newer models

• Anthropic: Claude 1.x, Claude 2.x, Claude 3.x, Claude 3.5 models

• Google: Gemini 1.0, Gemini 1.5, Gemini 2.0 models

• Other providers: Additional models as they become available

Managing model prices

You can view and manage model pricing through the Phoenix UI:

1. Navigate to Settings → Models in the Phoenix interface

2. View existing models and their pricing information

3. Add custom models or override pricing for existing models

4. Set different prices for prompt (input) and completion (output) tokens

Adding a custom model

To add pricing for a model not in the built-in table:

1. Click Add new model in the Models settings page

2. Fill in the model details:

   • Model Name: Human-readable name for the model

   • Name Pattern: Regex pattern to match the model name in traces

   • Provider: Model provider (optional)

   • Prompt (Input) Cost: Cost per 1M input tokens

   • Completion (Output) Cost: Cost per 1M output tokens

   • Start Date: When this pricing becomes effective (optional)

Advanced pricing configuration

For models with complex pricing structures, you can configure detailed token pricing:

• Prompt Price Breakdown: Different rates for cache_read, cache_write, audio, image, video tokens

• Completion Price Breakdown: Different rates for reasoning, audio, image tokens

• Provider Matching: Match models by provider to avoid naming conflicts

Viewing cost data

Once configured, Phoenix automatically displays cost information throughout the interface:

Trace-level costs

• Total cost for the entire trace

• Breakdown by prompt vs completion costs

Span-level costs

• Individual span costs with detailed breakdowns

• Token type-specific cost details

Session-level costs

• Aggregated costs across all traces within a session

• Session-based cost analysis for multi-turn conversations

• Cost tracking for extended user interactions

Experiment-level costs

Phoenix automatically tracks costs for traced experiments, providing detailed cost analysis across experiment runs:

• Total experiment cost: Sum of all LLM costs across all experiment runs

• Cost per experiment run: Individual cost for each dataset example run through an experiment Experiment costs are automatically calculated when you:

1. Run experiments on datasets through Phoenix

2. Include proper token count and model information in your traced LLM calls

3. Have model pricing configured for the models used in experiments

Project-level analytics

• Total costs across all traces in a project

• Cost trends over time (coming-soon)

• Most expensive models (coming-soon)