Skip to main content
Arize AX prices every LLM span when it ingests it, then adds the spans up per trace, so you can filter, monitor, and build dashboards on what each request costs. Default cost configs cover common models from OpenAI, Anthropic, Google, Mistral, and others. A custom cost config sets the price for a model that has no default or replaces a default price.
Cost is calculated at ingestion and is not recomputed later. Set up custom pricing before you send the traces you want priced.

​
How Arize AX calculates cost

  1. If the span already carries llm.cost.total, llm.cost.prompt, or llm.cost.completion, Arize AX keeps those values and skips the lookup.
  2. Otherwise it reads the model name from the first of these that is set: llm.model_name, the model_name key in llm.invocation_parameters, or the model_name key in metadata.
  3. It picks the cost config for that model. A config scoped to the span’s space wins over one scoped to its organization, which wins over an Arize default. Within a scope, an exact model name match beats a regex match, and a config whose provider matches llm.provider beats one with no provider.
  4. It multiplies each token count on the span by the matching token type’s price per 1M tokens in the config. Tiered pricing applies a different price once the token count passes a threshold.
Cost configs are cached for 10 minutes, so a new or edited config can take up to 10 minutes to apply to incoming spans.

​
Supported token count fields

Arize AX reads token counts from the OpenInference semantic conventions. Auto-instrumentation sets them. With manual instrumentation, set them yourself. You can add your own token types under prompt_details or completion_details. Arize AX prices any token count whose name matches a token type in the cost config.

​
Cost attributes on spans

Arize AX writes the result back to the span. Set these yourself to supply your own cost instead.

​
Use cost data

Cost attributes work anywhere span attributes do: filter traces or spans above a cost threshold, create monitors on high-cost traces, and build dashboards on specific token types. The trace view shows the summed cost of every LLM span in the request.
Trace-level cost aggregated across all LLM spans in a request in Arize AX
Each LLM span’s Attributes tab breaks the cost down by token type, which shows how much reasoning, cache, or image tokens contribute to an expensive step.
LLM span Attributes tab in Arize AX showing the llm.cost breakdown with prompt, completion, prompt_details, completion_details, and total

​
Next steps

Custom Cost Config

Set the price per token for a model that has no default, or replace a default price.

Customize your traces

Add the semantic conventions, including token counts, that Arize AX reads from your spans.