Policy adherence measures whether an AI system follows defined rules. Those rules might cover content, brand voice, legal constraints, security permissions, escalation paths, disclosure requirements, or which tools the system may call and when.
Most work labeled “measuring policy adherence” turns out to be policy authoring, because the policy does not exist in a checkable form yet. It exists as a slide, a paragraph in an onboarding doc, three sentences buried in a system prompt, and a shared understanding among four people. You cannot measure adherence to that. The first deliverable is a written set of rules, each naming the situation it applies to and the behavior it requires, in language specific enough that two reviewers reading one transcript reach the same verdict.
Key takeaways
- A policy has to be written as checkable criteria before adherence can be measured at all. Vague rules produce vague verdicts.
- Every rule needs four parts: when it applies, what is required or forbidden, what evidence shows it happened, and how bad a violation is.
- Stating a rule in a prompt is a request, not enforcement. Enforcement lives in guardrails, tool permissions, and code.
- Prefer deterministic checks. Reserve judge-based scoring for genuinely graded rules, and give it the policy text as its rubric.
- Report adherence per rule. A high aggregate can hide total failure on the one rule with legal consequences.
Turning a policy into checkable criteria
Take a rule as it usually arrives: “Be professional and don’t give financial advice.” Neither half is measurable. Professional means nothing an evaluator can apply consistently, and “financial advice” means one thing to your compliance team and another to everyone else. The rewrite gives each rule four components:
- Trigger. When the rule applies. Every response? Only when the user asks about pricing? Only in authenticated sessions?
- Requirement. The behavior, phrased as something observable. “Includes the disclosure sentence verbatim.” “Does not recommend a specific security by name.” “Escalates to a human when the user states they are in distress.”
- Evidence. What in the trace proves it: a string in the output, the absence of a category, a particular tool call, a routing decision, a span that exists or does not.
- Severity. Whether a violation blocks release, files a ticket, or gets counted. Treating every rule the same means the important ones drown.
Rules that survive this rewrite are testable. Rules that cannot be are aspirations, and labeling them that way beats pretending a score measures them. Testing against rules like these differs from conventional test suites, since the same input can produce a compliant response one run and a violation the next, which is one of the failure modes traditional software tests miss.
Enforcing is not the same as measuring
Rules can live in four places, and they are not interchangeable.
The prompt is where most policies start and the weakest place to keep them. A model instructed not to do something usually does not, which is different from cannot. Long policies make it worse, since instructions buried mid-prompt get followed less reliably than those at either end.
A policy layer or guardrail checks input and output against the rules and blocks, rewrites, or routes. This is real enforcement for content rules, and it costs latency.
Tool permissions and code are the only hard boundary. A rule that the agent must not issue refunds above a threshold is enforced by the refund API rejecting the call, not by the agent being told. Anything with real consequences belongs here.
Evals measure. They enforce nothing, and they are how you find out the other three layers are leaking.
Most teams need all four, split by consequence. Ask what happens if the model ignores this rule once. If the answer is serious, the rule cannot live only in a prompt.
Measuring adherence
Start with deterministic checks, because they are cheap, fast, and have no variance. A required disclosure either appears or it does not. A forbidden tool either was called or was not. An escalation either happened or it did not. Write as many rules in this form as the policy allows.
What remains is graded: tone, whether an explanation was adequate, whether a recommendation crossed from information into advice. These need a judge, and the policy text is the rubric. Give the scorer the rule as written, the trigger condition, and a few labeled examples on each side of the line, then check its verdicts against human review before trusting the number. The usual cautions about when a judge is the right instrument apply, and judges disagreeing on a rule almost always means the rule is ambiguous.
Whichever form the check takes, tie the verdict to the decision point rather than the session. Knowing a run violated the escalation rule is less useful than knowing which span, with which inputs and which retrieved context, is where escalation should have fired. Tracing an agent’s steps and evaluating them individually is what makes a violation debuggable instead of merely counted.
Where adherence programs go wrong
The policy contradicts itself. Two rules written by different teams collide in a case neither anticipated, and the model picks one. This surfaces as inconsistency, and the fix belongs in the document, not the prompt.
The policy outgrew the context. A policy that keeps accumulating rules stops fitting in a prompt the model reliably follows. Split it: hard rules to code and guardrails, situational rules retrieved only when their trigger fires.
The document and the shipped prompt drift apart. Someone edits the prompt to fix a bug, nobody updates the policy document, and the eval now measures adherence to a rule nobody implemented. Version them together.
Aggregate reporting hides the rule that matters. Report per rule and per severity tier. One number across all rules is comfortable and uninformative.
FAQ
How do I measure whether my AI system follows a policy?
Rewrite each rule with a trigger, an observable requirement, the evidence in the trace, and a severity. Implement every rule you can as a deterministic check, use a judge with the policy text as its rubric for the graded remainder, and validate that judge against human labels. Run it against a set that deliberately includes the situations triggering each rule, because ordinary traffic rarely exercises the interesting ones. Report results per rule.
What is the difference between policy adherence and compliance?
Policy adherence is whether the system follows the rules you wrote for it. Compliance is whether the system, and your evidence about it, satisfies external legal, regulatory, and contractual requirements. Adherence work supports compliance, since documented rules with measured results are the kind of evidence a review asks for, but the scopes differ and your policy may be stricter or looser than any external requirement.
Can I just put the policy in the system prompt?
For low-consequence rules, often yes. For anything with legal, financial, or safety weight, no. Prompt instructions are followed probabilistically, and adversarial input is aimed directly at that fact. Use the prompt to shape default behavior, and enforce the rules that matter in guardrails and in tool permissions.
What happens when two policy rules conflict?
The model resolves it silently and inconsistently, which is the worst available outcome. Conflicts show up as unstable verdicts on similar cases, so when adherence scores swing on near-identical inputs, look for competing rules before blaming the model. Fix it in the policy by giving rules an explicit precedence order.
How granular should policy rules be?
Granular enough that a rule fails for one reason. If a single rule can be violated by three different behaviors, you cannot act on the number.
