Report

AI governance in banking: How to scale AI agents

Banking agents can fail in ways that error dashboards miss: a credit memo cites the wrong figure, an advisor copilot overlooks a client’s mandate, or an alert is dismissed on incomplete evidence. Scaling those workflows requires a record of what happened and a way to review changes before they reach production.

This report gives bank AI, risk, data, and platform leaders a practical framework for observability, evaluation, and supervised agent improvement across federated business units. It connects local development with shared controls and deployment choices for data residency requirements.

Download the report to learn how to:

Instrument Phoenix locally in a VPC or on-prem so a pod can trace and evaluate without a platform ticket (or moving regulated data).
Stand up a three-tier fabric: Phoenix at the edge, ADB as the shared telemetry store, and Arize AX as an isolated control plane.
Run the 2026 improvement loop across Signal, diagnosis, managed-agent PRs, agent experiments, and online evals with human approval on every change.
Layer code evaluators, LLM judges, and agentic judges onto the three lines of defense, then version those judges as an MRM control.
Trace coding agent harnesses (Claude Code, Cursor, Codex) as a first-class channel instead of an unmonitored side door.

The federated agent control plane

01 PHOENIX OSS: LOCAL TRACING AND EVALS
02 OPENINFERENCE / OTEL INSTRUMENTATION
03 ARIZE DATABASE (ADB)
04 ARIZE AX CONTROL PLANE
05 SIGNAL AND MANAGED AGENTS
06 HUMAN APPROVAL AND AUDIT TRAIL

Get the report

Download the full paper for the federated architecture, 2026 improvement loop, layered eval model, obligation-to-evidence table, and 90-day operating plan.

PRODUCTION

AI agent use cases in banking

Each use case names the failure that matters and what the platform has to catch. Developer agents are often the easiest first instrumentation landing because the sessions are already happening.

Developer agents and legacy modernization

Coding-assistant sessions are an unmonitored channel touching source and secrets. Catch it with harness tracing for Claude Code, Cursor, and Codex, plus fleet-level token and action visibility.

KYC / AML alert triage

The agent dismisses an alert on incomplete evidence, or the reasoning cannot be shown to a regulator. Catch it with retrieval-completeness evals, trajectory review, and full decision replay per alert.

Wealth advisor copilot

Generic guidance presented as suitable advice for a specific client mandate. Catch it with prohibited-advice and groundedness judges, plus context-drift monitoring.

Credit memo drafting

A figure that is plausible, cited, and not what the source document says. Catch it with source-attribution evals and span-level provenance for every claim.

Contact center and voice assist

A mis-stated fee or term on a recorded call that supervision never samples. Catch it with voice session capture and replay, transcript evals, and interruption and latency metrics.

From obligation to evidence

The paper maps EU AI Act, SR 11-7 / CCAR, DORA, FINRA 3110 / MiFID II, GDPR, NIST AI RMF / ISO 42001, and OWASP LLM risks onto the spans, eval suites, and approval logs AX already produces.

THE PROBLEM

Why bank AI stalls between pilot and production

Global banks are not centralized technology firms. Wealth Management and Investment Banking run independent P&Ls, stacks, and regional mandates. A single monolithic AI platform becomes the bottleneck: local teams need to prototype inside their own VPC or on-premise estate without waiting on global provisioning.

Generative AI wants a whole-customer view. The data sits in mainframes, regional VPCs, and departmental warehouses under residency rules. Models built on partial context produce contextual hallucinations: answers that are factually correct and wrong for this client’s financial position.

A multi-step agent is harder to explain than traditional ML. The outcome depends on retrieval, tool arguments, and a reasoning path that differs run to run. Supervisors now expect reconstructive traceability: if an agent declines a facility or mis-states a product, you replay the exact path.

Production agents also fail in two distinct ways. Timeouts, tool errors, and 5xxs can be alerted on. Nuanced failures are not deterministic: wrong retrieval, bad tool arguments, skipped steps, plausible answers built on bad reasoning. The second class is where regulated risk lives, and it is the class manual spot checks miss.

THE CENTRAL OPERATING PRINCIPLE

Every consequential agent run should produce enough evidence to reconstruct the path: what was retrieved, which tools ran, which prompt and model versions were live, and whether the outcome stayed inside policy.

EVALUATION

Designing layered defenses to build better agents

No single evaluator is trustworthy on its own. Arize builds evals the way safety engineers build controls: cheap deterministic checks first, LLM judges layered on, agentic judges for trajectory quality, and human intervention where it counts.

Code evaluators

Schema violations, banned terms, PII patterns, tool-call errors, latency and cost budgets. Owned by business-unit engineering.

LLM judges

Hallucination, groundedness against retrieved sources, tone, prohibited advice, relevance. Owned by the central platform team.

Agentic judges

Trajectory failures, multi-step reasoning quality, and emerging failure modes nobody wrote an eval for. Owned by platform and model risk.

Human in the loop

Annotation queues for dispute cases, judge calibration, and sign-off on high-materiality releases. Owned by second line and audit.

“Technology moves at an exponential pace. Institutional trust moves at a human one.”

Jitendra Yadav Head of GenAI Solutions, Arize AI

Agents are already in production workflows at financial institutions. But most institutions haven't yet moved from working pilots to supervised, enterprise-wide systems.

Arize provides the control tower for agents including real-time visibility that platform teams, risk officers, and business leaders need to scale responsibly.

Keep reading

Recommended resources

See all resources

Papers

The agent reliability gap

Agent capability has advanced faster than the production systems around it. A strong model can still fail when…

From Signal to PR: What if your agents got better every time they failed?

Signal, a managed agent built into Arize AX, continuously reviews production traces, surfaces ranked issues with evidence and…

Read the post

Guide

How to build agent evals from traces

Evals are tests for AI; traces are logs for AI. This tutorial shows how to read agent traces,…

Read the guide

Docs

Start here

Build your playbook

See it in production

Company