Add cost tracking

Cost tracking is essential for understanding and managing the spend of your LLM-powered applications. Arize AX provides a flexible, powerful, and easy-to-configure system to track model usage costs across providers and model variants — whether you're using default pricing or defining custom rates.

Why is Cost Tracking Important?

Arize AX's cost tracking enables you to:

• Monitor LLM spend across use cases, models, and providers.

• Inspect token cost

• Track granular costs by token type (e.g., prompt, completion, audio, cache).

• Customize pricing to reflect discounted rates, fine-tuned models, or non-standard providers.

We currently support 63 default cost configurations for common model and provider combinations, making it easy to get started with no setup required in many cases.

If there is a default you'd like us to add reach out to [e-mail]

Using Cost Tracking in Arize AX

Once cost tracking is enabled, you can analyze and explore cost data at various levels of detail within the Arize AX UI to better understand and manage your LLM application spend.

Filtering and Monitoring

All cost attributes are available throughout the platform and can be used to:

• Filter traces or spans where cost exceeds a defined threshold

• Create monitors for high-cost traces or model behavior anomalies

• Build dashboards based on specific token types or cost groupings (e.g., prompt, audio, image)

Trace-Level Visualization

At the trace level, Arize aggregates cost across all LLM spans in the trace. This provides a complete view of how much it cost to serve a given request end-to-end, which is especially useful for understanding the impact of complex flows or multi-stage prompts.

Span-Level Visualization

You can also inspect cost at the individual span level, including a breakdown by token type. This allows you to:

• Pinpoint expensive steps in the LLM pipeline

• Analyze the relative contribution of different token categories (e.g., reasoning, cache, image)

Set up Cost Tracking

1. Use a Default (Zero Setup)

If your model and provider match a default, Arize automatically applies the correct pricing — no action needed.

2. Customize a Default

To tweak an existing config (e.g., apply discounts):

• Go to Settings > Cost Tracking > Configuration

• Click Options > Clone on a default config

• Edit fields like token type cost or provider name

3. Create from Scratch

To define your own model config:

• Click Add New

• Enter the model name (required)

• Optionally enter the provider

• Specify cost per 1 million tokens for each token type

• Assign each token type to Prompt or Completion

Cost configs are saved at the organization level.

Cost Tracking Settings

• How Cost Tracking Works

• Supported Token Types and Semantic Conventions