European teams keep describing the same gap: a Responsible AI (RAI) policy written two or three years ago, an engineering team shipping agents, and almost nothing connecting the two. The EU AI Act makes that gap operational: teams may need to show how a named system behaved, changed, and was reviewed over time. For developers, that means traces and evaluations. For product managers, it means owners, thresholds, and release decisions backed by evidence.
Disclaimer: This is my own analysis, written from an engineering perspective. It is not legal advice, formal guidance, or a statement of Arize’s position. The dates and obligations below reflect my reading of the EU AI Act and the Digital Omnibus as of August 2026, and the details are still moving. Verify anything you plan to act on against the Official Journal and with your own counsel.
The practical shift: principles become evidence
Read Chapter III and you won’t find any values you haven’t already published. Fairness, transparency, human oversight, robustness, privacy, accountability. Your policy probably uses five of those words.
What’s different is who the audience is. A principle written for your own organisation can stay a principle. A principle that a national authority might ask you to demonstrate, on a named system, on a Tuesday in 2028, has to turn into a number with a history behind it.
Here is one way to translate those six principles into operational artifacts. Most of the evidence comes from ordinary AI engineering: traces, evaluations, annotations, CI, and audit logs.
Most of the right-hand column is not specialized compliance tooling. It is the ordinary equipment of a team that takes evaluations seriously. That’s why I keep arguing that the RAI programme and the evaluation programme should be one program. Product and engineering teams should agree on the metric, owner, threshold, review path, and release consequence for each principle.
One warning before trusting the metrics
There’s a failure mode I’ve now seen enough times to be blunt about it. Someone stands up an LLM judge, points it at a bias criterion, gets a score, and puts that score in a risk committee deck. Nobody ever checked whether the judge agrees with a human on the cases that actually matter.
That’s worse than having no metric, because it makes people confident when they shouldn’t be. Before a fairness score leaves your team, measure judge-to-human agreement on a representative calibration set, inspect the disagreement cases, and retain that set as models and traffic drift. Raw accuracy alone is not enough. This work is tedious, but it separates an operational programme from a decorative one.
Why sixteen more months are not a reason to wait
The obvious reading of the Omnibus is that Annex III teams can relax until late 2027. The bit that worries me is Article 111.
Systems already on the market get a transition period, but only if they aren’t significantly modified. That was drafted with the assumption of software that ships a couple of times a year. It doesn’t describe an agent.
So ask your team this: over the last quarter you changed the system prompt, rebuilt the retrieval index, moved to a newer model, and added two tools. Which of those was the significant modification? Most teams can’t answer because nobody recorded the changes in a form anyone can review.
The consistent advice I have seen is to use the extra time, not wait it out. Sixteen months of policy drafting will not create sixteen months of evidence. Instrumentation will.
Make documentation part of the system
Most of the high-risk chapter stops sounding legal once you read it as an engineer. Article 12 wants records. Article 14 wants a person who can understand the system and step in. Article 15 wants accuracy and robustness you can show. Article 72 wants you to keep looking after release.
These are telemetry, evaluation, workflow, and release-management requirements. You cannot reconstruct them reliably after the fact because the Act asks for a history. The system has to write that history while work happens.
| Obligation | The artifact that answers it |
|---|---|
| Art. 12 Record-keeping | OpenInference/OpenTelemetry traces across the whole agent trajectory: model calls, tool arguments, retrieval results, handoffs. With retention you’ve chosen deliberately. |
| Art. 14 Human oversight | Annotations on spans using a shared schema, plus queues that route failures to reviewers. Each label carries the reviewer and the time. |
| Art. 15 Accuracy and robustness | Evaluators offline on datasets and online against live traffic, wired into CI so a regression doesn’t reach users. |
| Art. 10 Data governance, bias examination | Masking and PII redaction inside your own process, before anything is exported. Disparity metrics on production traffic, not only on the training set. |
| Art. 50 Transparency | An evaluator confirming the disclosure was actually there. A policy requires disclosure; this shows it happened. |
| Art. 72 Post-market monitoring | Monitors and alerts on the same metrics you gate releases with, so quality means one thing before and after deployment. |
| Art. 11 / Annex IV Technical documentation | Versioned prompts, dataset lineage, experiment results. The change history of your technical file is meant to summarise. |
Usual caveat: this is engineering guidance. How your systems get classified, and which conformity route you take, is a conversation for your counsel.
A reference architecture for evidence
The mapping above only works if the plumbing produces reliable, reviewable evidence. With Arize AX, the reference architecture looks like this.
Agent runtime, then a redacting span processor running inside your application, then an OTel collector, then storage (EU region or your own cluster), then evaluators online and offline, then a labeling queue for the failures, then a benchmark dataset, then a CI gate on the next release, then monitors and audit logs.
The position of that third component is the part people get wrong. If redaction happens in your process rather than at the far end, you stop having to choose between a complete record and data minimisation. You get both.
| Layer | What runs there | What it leaves behind | Principle |
|---|---|---|---|
| 1. Instrumentation | OpenInference and OpenTelemetry instrumentation, plus a span processor that masks attribute classes and redacts PII by regex or with Presidio | A complete but minimised record of every run | Privacy (12, 10) |
| 2. Storage and residency | The Arize AX EU region in Belgium, or self-hosted AX on your own cluster and object storage, or an air-gapped install | History under a known jurisdiction, one boundary to audit | Privacy (12, GDPR) |
| 3. Evaluation | Online evaluators on live traffic, offline evaluators over datasets, deterministic checks alongside LLM judges, with judge-to-human agreement measured | Fairness, safety, groundedness and disclosure metrics over time, and a reason to believe them | Fairness, robustness, transparency (15, 10, 50) |
| 4. Human review | Annotation configs defining the schema, queues routing failures to named reviewers | Who reviewed what, what they decided, when | Human oversight (14) |
| 5. Change control | Benchmark datasets in CI (GitHub Actions, GitLab, Jenkins, Azure DevOps), production monitors, audit logs on access and export | A release history with reasons, drift alerts, an access trail | Accountability (15, 72, 11) |
Here are two defaults to set deliberately:
- Retention. Debugging defaults assume a short memory. Regulatory evidence requires a deliberate horizon. Set trace and annotation retention against the obligations that apply to the use case, and verify the annotation lookback window before relying on last quarter’s traces.
- Sampling. Sampling is normal in observability but dangerous for record-keeping. Sample low-risk traffic where appropriate. Keep high-risk workflows at full fidelity unless your risk and legal owners approve another approach. A fairness score computed on 5% of traffic may be useful, but it is not the same as a complete record.
One agent, end to end
A creditworthiness assistant is a good example because it’s squarely Annex III and it’s also the oldest problem in Responsible AI: an automated decision about someone’s access to money, explained by a model.
Suppose a customer asks why their limit was set where it was. The agent retrieves policy documents, calls a scoring service, and drafts an explanation. The surrounding workflow should do five things:
- The trace is captured, already redacted. Prompt, retrieved policy text, tool call, output, all of it, with names and account numbers replaced before the span leaves the process. Complete record, no personal data in transit.
- Evaluators score it. Groundedness against the policy that was actually retrieved. Outcome disparity across applicant cohorts. Whether the Article 50 disclosure was present.
- Failures go to a person. Low-groundedness traces land in a queue. An analyst confirms or overturns the explanation, and their name and the timestamp attached to the span. That’s the Article 14 evidence, and note that nobody generated it as evidence. They generated it by doing their job.
- The failures become the gate. Reviewed traces turn into a benchmark dataset, and the next prompt or model change runs against it in CI. A regression fails the build, which means every release has a documented reason behind it.
- Monitoring continues. Alerts on drift in the same metrics. Audit logs covering logins, changes, and every export, including which project the data came from and who pulled it.
No step exists only for the regulator. It is the same loop you would build to make the agent more reliable and the product easier to operate.
Decide where the data lives before you instrument anything
Traces of a credit conversation are personal data. So, solving your AI Act problem by creating a GDPR problem isn’t progress.
Arize AX supports three deployment patterns, in increasing order of control: an EU region in Belgium; self-hosting on your Kubernetes cluster and object storage, where Arize stores nothing; and an air-gapped install with no outbound path for classified or heavily regulated environments. The choice affects residency, operational ownership, and the security boundary you must audit.
Across all three: SOC 2 Type II, ISO 27001, PCI DSS, HIPAA, GDPR, SAML 2.0 SSO against your own IdP, role-based access down to individual projects.
What Arize AX doesn’t do
Some of this belongs to other people. We don’t decide whether your system counts as high-risk. We don’t run your conformity assessment, put a CE mark on anything, file your registration, or build the quality management system Article 17 asks for. Those are calls someone has to make, not things you can measure, and they sit with your governance people and your lawyers.
Your fundamental rights assessment is on that list too, though I want to push on it a bit. Most of the Article 27 assessments I’ve seen are built on what people assume the system does. The disparity numbers and the reviewed traces are what it actually did. You still have to make the judgement call. You just don’t have to guess at what you’re judging.
There’s also something we genuinely can’t do yet. We don’t give you tamper-proof records. Traces, evals and annotations will show an auditor what happened and who signed off, but nobody can prove afterwards that the file wasn’t edited. The standards for that are still being written. I’d rather tell you now than have you work it out halfway through an audit.
And then there’s the bit no vendor can sell you at all. A governance platform will keep your risk register and your attestations tidy. It won’t tell you what your agent did at 04:12 on a Tuesday, whether anyone looked at it, or whether last week’s prompt change made things worse. Only your own instrumentation knows that.
Don’t ship vibes. Least of all to a regulator.
December 2027 is sixteen months of traces you will either have or you will not. Start with one Annex III use case. Identify the principle you would struggle most to demonstrate today, define the metric and owner, instrument the workflow, and make the result part of the release process.
Book a demo · Self-host Arize AX · Arize AX on SaaS · Read the tracing docs
Engineering practice, not legal advice. The dates here reflect the Digital Omnibus on AI as in force on 27 July 2026 and are worth checking against the Official Journal and your own counsel before you rely on them.