Developer agents and legacy modernization
Coding-assistant sessions are an unmonitored channel touching source and secrets. Catch it with harness tracing for Claude Code, Cursor, and Codex, plus fleet-level token and action visibility.
Report
AI governance in banking: How to scale AI agents
Banking agents can fail in ways that error dashboards miss: a credit memo cites the wrong figure, an advisor copilot overlooks a client’s mandate, or an alert is dismissed on incomplete evidence. Scaling those workflows requires a record of what happened and a way to review changes before they reach production.
This report gives bank AI, risk, data, and platform leaders a practical framework for observability, evaluation, and supervised agent improvement across federated business units. It connects local development with shared controls and deployment choices for data residency requirements.
Download the report to learn how to:
- Instrument Phoenix locally in a VPC or on-prem so a pod can trace and evaluate without a platform ticket (or moving regulated data).
- Stand up a three-tier fabric: Phoenix at the edge, ADB as the shared telemetry store, and Arize AX as an isolated control plane.
- Run the 2026 improvement loop across Signal, diagnosis, managed-agent PRs, agent experiments, and online evals with human approval on every change.
- Layer code evaluators, LLM judges, and agentic judges onto the three lines of defense, then version those judges as an MRM control.
- Trace coding agent harnesses (Claude Code, Cursor, Codex) as a first-class channel instead of an unmonitored side door.
Arize AI
SCALING GOVERNED AI AGENTS
How tier-one banks run an auditable agent improvement loop across federated business units.
Framework The federated agent control plane
- 1 PHOENIX OSS: LOCAL TRACING AND EVALS
- 2 OPENINFERENCE / OTEL INSTRUMENTATION
- 3 ARIZE DATABASE (ADB)
- 4 ARIZE AX CONTROL PLANE
- 5 SIGNAL AND MANAGED AGENTS
- 6 HUMAN APPROVAL AND AUDIT TRAIL
PRODUCTION
AI agent use cases in banking
Each use case names the failure that matters and what the platform has to catch. Developer agents are often the easiest first instrumentation landing because the sessions are already happening.
KYC / AML alert triage
The agent dismisses an alert on incomplete evidence, or the reasoning cannot be shown to a regulator. Catch it with retrieval-completeness evals, trajectory review, and full decision replay per alert.
Wealth advisor copilot
Generic guidance presented as suitable advice for a specific client mandate. Catch it with prohibited-advice and groundedness judges, plus context-drift monitoring.
Credit memo drafting
A figure that is plausible, cited, and not what the source document says. Catch it with source-attribution evals and span-level provenance for every claim.
Contact center and voice assist
A mis-stated fee or term on a recorded call that supervision never samples. Catch it with voice session capture and replay, transcript evals, and interruption and latency metrics.
From obligation to evidence
The paper maps EU AI Act, SR 11-7 / CCAR, DORA, FINRA 3110 / MiFID II, GDPR, NIST AI RMF / ISO 42001, and OWASP LLM risks onto the spans, eval suites, and approval logs AX already produces.
THE PROBLEM
Why bank AI stalls between pilot and production
Global banks are not centralized technology firms. Wealth Management and Investment Banking run independent P&Ls, stacks, and regional mandates. A single monolithic AI platform becomes the bottleneck: local teams need to prototype inside their own VPC or on-premise estate without waiting on global provisioning.
Generative AI wants a whole-customer view. The data sits in mainframes, regional VPCs, and departmental warehouses under residency rules. Models built on partial context produce contextual hallucinations: answers that are factually correct and wrong for this client’s financial position.
A multi-step agent is harder to explain than traditional ML. The outcome depends on retrieval, tool arguments, and a reasoning path that differs run to run. Supervisors now expect reconstructive traceability: if an agent declines a facility or mis-states a product, you replay the exact path.
Production agents also fail in two distinct ways. Timeouts, tool errors, and 5xxs can be alerted on. Nuanced failures are not deterministic: wrong retrieval, bad tool arguments, skipped steps, plausible answers built on bad reasoning. The second class is where regulated risk lives, and it is the class manual spot checks miss.
THE CENTRAL OPERATING PRINCIPLE
Every consequential agent run should produce enough evidence to reconstruct the path: what was retrieved, which tools ran, which prompt and model versions were live, and whether the outcome stayed inside policy.
EVALUATION
Designing layered defenses to build better agents
No single evaluator is trustworthy on its own. Arize builds evals the way safety engineers build controls: cheap deterministic checks first, LLM judges layered on, agentic judges for trajectory quality, and human intervention where it counts.
Code evaluators
Schema violations, banned terms, PII patterns, tool-call errors, latency and cost budgets. Owned by business-unit engineering.
LLM judges
Hallucination, groundedness against retrieved sources, tone, prohibited advice, relevance. Owned by the central platform team.
Agentic judges
Trajectory failures, multi-step reasoning quality, and emerging failure modes nobody wrote an eval for. Owned by platform and model risk.
Human in the loop
Annotation queues for dispute cases, judge calibration, and sign-off on high-materiality releases. Owned by second line and audit.
"Technology moves at an exponential pace. Institutional trust moves at a human one."Jitendra Yadav // Head of GenAI Solutions, Arize AI
Agents are already in production workflows at financial institutions. But most institutions haven't yet moved from working pilots to supervised, enterprise-wide systems.
Arize provides the control tower for agents including real-time visibility that platform teams, risk officers, and business leaders need to scale responsibly.
Improve banking agents with controls you can audit
Download the report to get a banking reference architecture, evaluation framework, governance evidence map, and 90-day rollout plan.
Keep reading
See all resources
Recommended resources
Papers
The agent reliability gap
Agent capability has advanced faster than the production systems around it. A strong model can still fail when…
Read more
Blog
From Signal to PR: What if your agents got better every time they failed?
Signal, a managed agent built into Arize AX, continuously reviews production traces, surfaces ranked issues with evidence and…
Read the post
Guide
How to build agent evals from traces
Evals are tests for AI; traces are logs for AI. This tutorial shows how to read agent traces,…
Read the guideDon’t ship vibes.
Arize gives AI teams observability and evals to understand and improve agent performance.
