Cost optimization for AI agents

Find wasted spend in your agent runs.

Cost Agent investigates production traces, finds recurring sources of unnecessary spend, and recommends fixes with estimated savings.

Get started free Book a demo

Cost Issues

Activetravel-agent/booking-api

Cost over time$4,050

Oct 9Oct 24Nov 8

Tokens over timeInput 448MOutput 13.3M

Oct 9Oct 24Nov 8

Cost by model

claude-opus-5

claude-haiku-5

$0$2k$4k

Issues found3 issues

Potential savings
$1,789/month

Itinerary calls resend the full trip history (~18k tokens) High · Last seen Nov 8

Airport-code lookups run on claude-opus-5 Medium · Last seen Nov 5

Itinerary calls resend the full trip history (~18k tokens)

Mark as fixed

claude-opus-5 · High · Found Nov 2, 9:27 AM · Last seen Nov 8, 4:01 PM

Create EvaluatorAdd to DatasetGitHub

Overview

claude-opus-5 is the dominant cost driver on travel-agent-prod. Every itinerary update resends the accumulated tool history, about 18,000 prompt tokens, for a reply under 200 tokens.

Evidence

claude-opus-5: 93% of project spend, 18.4k prompt tokens per itinerary call
Completions average 184 tokens, so the prompt is the cost
Sampled 3 itinerary spans: 17.9k to 18.4k prompt tokens each

Proposed fix

Keep the booking fields, drop the resolved tool history. Prompt size falls roughly in proportion to cost because the completions are already small.

$400K

in projected savings found in one week by a leading work management platform.

Analyze spend
Find recurring waste
Review potential savings

Repeated calls and oversized prompts add up.

Overpowered models

“Is SFO a valid airport code?”
1. LLMfrontier modela lookup
Expensive models handling work that a smaller model could perform.
Repeated work

“Book the hotel.”
1. Toolbook_hotel
2. Toolbook_hotelretry
3. Toolbook_hotelretry
Duplicate calls, unnecessary tool use, and runaway retries.
Expensive context

“Add Tuesday’s flight.”
1. LLM18,200 prompt tokenshistory repeated
Oversized prompts and repeated context increasing token consumption.

Find the savings in your traces.

Agent runslast 30 days

itinerary_update18.4kPrevious tool history
hotel_booking6.1k
itinerary_update17.9kPrevious tool history
trip_summary4.8k
airport_lookup1.2k
itinerary_update18.2kPrevious tool history
seat_selection2.3k

Recurring cause3 of 11,000

Every itinerary update resends the full trip history.

trip_4417 · a91c
trip_5230 · 6b04
trip_5861 · e27f

Proposed changeKeep booking fields. Trim old tool history.

Ranked by savingsIllustrative example

1Repeated trip historyEst. $1,485/month
Evidence
11,000 itinerary calls
Prompt tokens
18.4k → 9.4k per call
Estimated savings
$1,485/month
Next step
Validate task quality
2Airport-code lookups on claude-opus-5Est. $235/month
3book_hotel retries on timeoutEst. $69/month

Cost Agent

Find the cause of high costs. Review a change.

Measure

See what drives spend.

Break down recorded costs by model, task, and time window.

Cost

travel-agent-prodLast 30 days

Total spend$4,05030 days

Prompt tokens448M97% of all tokens

Cost per call$0.04786k calls

Cost over time$4,050

Oct 9Oct 24Nov 8

Task	Model	Prompt tokens / call	Spend
Itinerary update	claude-opus-5	18.4k	$3,190
Hotel booking	claude-opus-5	6.1k	$350
Airport-code lookup	claude-opus-5	1.2k	$250
Trip summary	claude-haiku-5	4.8k	$150
Seat selection	claude-haiku-5	2.3k	$60
Fare rules Q&A	claude-haiku-5	3.1k	$50

Investigate

Find recurring waste.

Investigate expensive models, repeated calls, retries, and inefficient execution patterns.

Cost Issues

Active

Issues found3 issues

Potential savings
$1,789/month

Itinerary calls resend the full trip history (~18k tokens)High · Last seen Nov 8

Airport-code lookups run on claude-opus-5Medium · Last seen Nov 5

book_hotel retries up to 3 times on timeoutMedium · Last seen Nov 7

Itinerary calls resend the full trip history (~18k tokens)

Mark as fixed

claude-opus-5 · High · Found Nov 2, 9:27 AM · Last seen Nov 8, 4:01 PM

Create EvaluatorAdd to DatasetGitHub

Overview

claude-opus-5 is the dominant cost driver, and its itinerary calls carry very large prompts: about 18,000 prompt tokens for 150 to 200 completion tokens.

Evidence

claude-opus-5: $126/day, 870 calls/day, $0.145/call, 93% of project spend
Last 7 days: $134/day, with itinerary prompts steady at 18.4k tokens
Sampled 3 itinerary spans: 17.9k to 18.4k prompt tokens, completions 177 to 184

Sample traces

trip_4417·itinerary·a91c
trip_5230·itinerary·6b04
trip_5861·itinerary·e27f

Recommend

Test cost savings against task quality.

Review suggested routing or configuration fixes, then use evaluations and experiments to verify cost reductions against your quality requirements.

Recommended change

1 of 3

Trim the repeated tool history on itinerary calls

Approve change

claude-opus-5 · travel-agent-prod · Large savings

Change

Field	From	To
Prompt context	Full tool history	Booking fields only
Prompt tokens / call	18.4k	9.4k
Est. weekly cost	$744	$397

Verified against quality

Replayed as an experiment on 212 saved itinerary examples before the change is applied.

Evaluator	Before	After	Result
Itinerary correctness	0.94	0.94	Hold
Booking completeness	0.97	0.96	Hold

Cost

travel-agent-prodLast 30 days

Total spend$4,05030 days

Prompt tokens448M97% of all tokens

Cost per call$0.04786k calls

Cost over time$4,050

Oct 9Oct 24Nov 8

Task	Model	Prompt tokens / call	Spend
Itinerary update	claude-opus-5	18.4k	$3,190
Hotel booking	claude-opus-5	6.1k	$350
Airport-code lookup	claude-opus-5	1.2k	$250
Trip summary	claude-haiku-5	4.8k	$150
Seat selection	claude-haiku-5	2.3k	$60
Fare rules Q&A	claude-haiku-5	3.1k	$50

Cost Issues

Active

Issues found3 issues

Potential savings
$1,789/month

Itinerary calls resend the full trip history (~18k tokens)High · Last seen Nov 8

Airport-code lookups run on claude-opus-5Medium · Last seen Nov 5

book_hotel retries up to 3 times on timeoutMedium · Last seen Nov 7

Itinerary calls resend the full trip history (~18k tokens)

Mark as fixed

claude-opus-5 · High · Found Nov 2, 9:27 AM · Last seen Nov 8, 4:01 PM

Create EvaluatorAdd to DatasetGitHub

Overview

claude-opus-5 is the dominant cost driver, and its itinerary calls carry very large prompts: about 18,000 prompt tokens for 150 to 200 completion tokens.

Evidence

claude-opus-5: $126/day, 870 calls/day, $0.145/call, 93% of project spend
Last 7 days: $134/day, with itinerary prompts steady at 18.4k tokens
Sampled 3 itinerary spans: 17.9k to 18.4k prompt tokens, completions 177 to 184

Sample traces

trip_4417·itinerary·a91c
trip_5230·itinerary·6b04
trip_5861·itinerary·e27f

Recommended change

1 of 3

Trim the repeated tool history on itinerary calls

Approve change

claude-opus-5 · travel-agent-prod · Large savings

Change

Field	From	To
Prompt context	Full tool history	Booking fields only
Prompt tokens / call	18.4k	9.4k
Est. weekly cost	$744	$397

Verified against quality

Replayed as an experiment on 212 saved itinerary examples before the change is applied.

Evaluator	Before	After	Result
Itinerary correctness	0.94	0.94	Hold
Booking completeness	0.97	0.96	Hold

Book a demo

Arize AX

Managed Agents

Automate recurring AI engineering work.

Cost Agent is an Arize managed agent.

Start from a ready-made agent for the job, or build your own with access to your traces, code, and tools.

Signal

Find and investigate agent failures.

Reviews production traces on a schedule, groups recurring failures into ranked issues, and proposes fixes for review.

Learn more

Eval Health

Find gaps and problems in your evals.

Finds missing coverage, unreliable evals, and unnecessary evaluation costs, then recommends fixes for review.

Learn more

Response toxicity has no eval on this project

CoverageMute✓ Mark as fixed

response-toxicity · Found on Nov 2, 9:27 AM · Last seen Nov 8, 4:01 PM

Suggested actions

Create evalCreate eval "Response Toxicity" and run it on this project▾ View details

Dismiss✓ Accept and run

Overview

The project has no toxicity eval, so none of its 1,240 user-facing LLM spans are judged for harmful content before it reaches a traveler.

Plan

Eval — adapted from the toxicity product template. Labels: toxic=1, not_toxic=0. Direction: minimize. Granularity: span.

Exact-match check is running as an LLM judge

EfficiencyMute✓ Mark as fixed

exact-match-v2 · Found on Nov 3, 7:12 AM · Last seen Nov 7, 6:30 PM

Suggested actions

Convert to code evalReplace the LLM judge on "Exact Match" with a code eval▾ View details

Dismiss✓ Accept and run

Overview

This evaluator checks whether the answer matches a fixed string but calls an LLM every time. Code returned the same label on all 412 sampled spans.

Plan

Code eval — compares output.value against expected.value. Labels: match=1, no_match=0. Spend after the change: $0, from $1.30 per 1k runs.

Hallucination eval gives no signal

EffectivenessMute✓ Mark as fixed

hallucination-v1 · Found on Oct 26, 3:40 PM · Last seen Nov 8, 11:12 AM

Suggested actions

Detach evaluatorDetach "hallucination-v1" from this project▾ View details

Dismiss✓ Accept and run

Overview

hallucination-v1 returned the same label on all 100 sampled spans, so it never separates a good answer from a bad one on this project.

Plan

Detach — stop the evaluator and the $8.40 per 1k runs it costs. Re-attach once its prompt separates factual from hallucinated on a labeled set.

Templates

Start from a template, or build your own.

Managed Agents include starting points for debugging, incident response, cost analysis, and recurring health checks.

Pick one and adapt it to your workflow, or start from scratch.

Connect the agent to the traces, code, and tools it needs for the task.

HarnessesClaude CodeCodexCursorOpenCodeHermesOpenClaw

SandboxesArizeClaude Managed AgentsVercel SandboxDaytona

SkillsGitHubArizeDynatraceDatadogCustom skill

Agent StudioAlyx

Describe a task, then choose an agent config to reuse a saved setup, or configure the managed agent manually.

Configuration

Harness*Claude CodeThe harness that executes your task

Sandbox*ArizeThe compute environment for the session

AI Provider*Arize managed (default)

Agent roleManaged Agent (Default)The permissions the run gets, separate from the harness and the skills

RepoSelect a repository

Skills+ Add skill

Task*

Describe the task — e.g. “Trace the checkout agent and flag any tool-call failures from the last 24h.”

Set up as automation

Book a demo

Find your next opportunity to reduce AI costs.

Review recommended changes and estimated savings, then test the impact on task quality before you ship.

Book a demo

Docs

Start here

Build your playbook

See it in production

Company

Find wasted spend in your agent runs.

Cost Issues

Itinerary calls resend the full trip history (~18k tokens)

Overview

Evidence

Proposed fix

Analyze spend

Find recurring waste

Review potential savings

Repeated calls and oversized prompts add up.

Find the savings in your traces.

Find the cause of high costs. Review a change.

See what drives spend.

Cost

Find recurring waste.

Cost Issues

Itinerary calls resend the full trip history (~18k tokens)

Overview

Evidence

Sample traces

Test cost savings against task quality.

Recommended change

Trim the repeated tool history on itinerary calls

Change

Verified against quality

Automate recurring AI engineering work.

Find and investigate agent failures.

Find gaps and problems in your evals.

Response toxicity has no eval on this project

Overview

Plan

Exact-match check is running as an LLM judge

Overview

Plan

Hallucination eval gives no signal

Overview

Plan

Start from a template, or build your own.

Find your next opportunity to reduce AI costs.

Subscribe to The Evaluator