Prioritize recurring failures.
Related failures are grouped into ranked issues, so your team knows where to start.
Production investigation for AI agents
Find agent failures.
Review proposed fixes.
Signal reviews production traces, ranks recurring issues, and drafts fixes for your team to review and ship.
~60%
of Signal's proposed fixes on Alyx, Arize's production agent, merged with no extra work.
-
Find issues automatically
Signal reviews traces on your schedule and groups recurring failures into ranked issues.
-
Investigate the cause
See what went wrong, which runs were affected, and what Signal recommends changing. Each investigation includes supporting traces.
-
Review a proposed fix
Connect your GitHub repository on Enterprise so Signal can open pull requests with proposed fixes. Your team reviews, tests, and merges the changes.
Agent failures don’t all look alike.
-
Wrong result
“Where’s my refund for order 4417?”
- LLMRead request
- Toolsearch_kb
- LLMAnswer: refunds take 5–7 daysmisses the goal
A plausible answer misses the user’s goal or relies on outdated information.
-
Wrong path
“Cancel my order”
- Toollookup_order
- Toolupdate_shippingwrong tool
- LLMAnswer: shipping address updatedorder never cancelled
The agent picks the wrong tool, takes a detour, or gets stuck.
-
Wasted work
“What’s my order status?”
- Toolget_order
- Toolget_orderduplicate call
- LLMAnswer: shipped Oct 8
Unnecessary calls add latency and cost, even when the task completes.
Which ones keep happening?
Group recurring failures into ranked issues.
new tracesspan OK
Scheduled reviewevery 6 h
Ranked issueslast 30 days
Arize Signal
Investigate issues. Review proposed fixes.
Investigate the likely cause.
Each issue comes with supporting traces, a likely cause, and a proposed fix.
Review a pull request with a proposed fix.
Attach a GitHub repo and Signal opens a pull request for your team to review and merge.
Signal is super valuable. We were calling our retrieval API unnecessarily on every turn, and at our scale, that was costing us real money.
Signal surfaced the pattern automatically, and that’s exactly the kind of hidden cost we needed to find and fix.
Managed Agents
Automate recurring AI engineering work.
Signal is an Arize managed agent.
Start from a ready-made agent for the job, or build your own with access to your traces, code, and tools.
Eval Health
Find gaps and problems in your evals.
Finds missing coverage, unreliable evals, and unnecessary evaluation costs, then recommends fixes for review.
Learn moreResponse toxicity has no eval on this project
CoverageMute✓ Mark as fixedresponse-toxicity · Found on Nov 2, 9:27 AM · Last seen Nov 8, 4:01 PM
Suggested actions
+
Create evalCreate eval "Response Toxicity" and run it on this project▾ View details
Dismiss✓ Accept and runOverview
The project has no toxicity eval, so none of its 1,240 user-facing LLM spans are judged for harmful content before it reaches a traveler.
Plan
Eval — adapted from the toxicity product template. Labels: toxic=1, not_toxic=0. Direction: minimize. Granularity: span.
Exact-match check is running as an LLM judge
EfficiencyMute✓ Mark as fixedexact-match-v2 · Found on Nov 3, 7:12 AM · Last seen Nov 7, 6:30 PM
Suggested actions
+
Convert to code evalReplace the LLM judge on "Exact Match" with a code eval▾ View details
Dismiss✓ Accept and runOverview
This evaluator checks whether the answer matches a fixed string but calls an LLM every time. Code returned the same label on all 412 sampled spans.
Plan
Code eval — compares output.value against expected.value. Labels: match=1, no_match=0. Spend after the change: $0, from $1.30 per 1k runs.
Hallucination eval gives no signal
EffectivenessMute✓ Mark as fixedhallucination-v1 · Found on Oct 26, 3:40 PM · Last seen Nov 8, 11:12 AM
Suggested actions
+
Detach evaluatorDetach "hallucination-v1" from this project▾ View details
Dismiss✓ Accept and runOverview
hallucination-v1 returned the same label on all 100 sampled spans, so it never separates a good answer from a bad one on this project.
Plan
Detach — stop the evaluator and the $8.40 per 1k runs it costs. Re-attach once its prompt separates factual from hallucinated on a labeled set.
Cost Agent
Find ways to reduce AI costs.
Finds unnecessary spend in model choices, repeated calls, and context, then recommends changes with estimated savings.
Learn moreTemplates
Start from a template, or build your own.
Managed Agents include starting points for debugging, incident response, cost analysis, and recurring health checks.
Pick one and adapt it to your workflow, or start from scratch.
Connect the agent to the traces, code, and tools it needs for the task.
HarnessesClaude CodeCodexCursorOpenCodeHermesOpenClaw
SandboxesArizeClaude Managed AgentsVercel SandboxDaytona
SkillsGitHubArizeDynatraceDatadogCustom skill
Agent StudioAlyx
Describe a task, then choose an agent config to reuse a saved setup, or configure the managed agent manually.
Configuration
Harness*Claude CodeThe harness that executes your task
Sandbox*ArizeThe compute environment for the session
AI Provider*Arize managed (default)
Agent roleManaged Agent (Default)The permissions the run gets, separate from the harness and the skills
RepoSelect a repository
Skills+ Add skill
Task*
Describe the task — e.g. “Trace the checkout agent and flag any tool-call failures from the last 24h.”
Set up as automationFind your first issue with Signal.
Signal is included on every Arize AX plan.
Turn it on in a tracing project and review your first ranked issues.