Policy-driven agents are agents whose behavior is governed by explicit policies rather than prompt instructions alone. Policies can cover tool permissions, data access, escalation rules, safety constraints, budgets, and deployment actions. The word doing the work is explicit: the rule exists somewhere you can point to, with a name, an owner, and a test, instead of living as a sentence inside a prompt that has been edited nine times by four people.
This is a design approach rather than a component. It starts from a question about every behavior you care about: is this something the model should decide, or something we have already decided? If the team has already decided, encode it outside the model, expose the decision in traces, and evaluate whether the agent followed it. If the team has not decided, the model gets to choose, and you should stop pretending the prompt sentence was a rule.
Key takeaways
- A policy is explicit when you can name it, say where it is enforced, and show whether the agent followed it on a given run. Anything failing those three tests is a preference, not a policy.
- Declaring a rule separates two jobs that prompts usually blur: deciding what should happen, and getting a model to produce good language.
- Rules written as data can be reviewed by the domain expert who owns them, which is rarely the person who owns the prompt.
- Most behavior is not policy-shaped. Judgment, phrasing, and novel situations belong to the model, and forcing them into rules produces an agent that refuses ordinary work.
What makes an agent policy-driven
Take a support agent with a refund limit. In the prompt-driven version, a sentence somewhere says not to approve refunds above a threshold. In the policy-driven version, the limit is a named rule with a value, the refund tool rejects calls above it, the trace shows the rule that fired, and a test case confirms the agent’s behavior at the boundary. Same intent, four differences that matter: the rule is findable, changeable without touching the prompt, enforceable, and checkable.
The practical test is three questions asked about a system already in production:
- Can you list the rules the agent operates under? If the answer requires reading a prompt and interpreting it, the rules are not declared.
- For each rule, where is it enforced? In code, in a tool boundary, in a retrieval filter, in a guardrail, or only in the model’s cooperation.
- Can you tell from a run whether a rule applied and what it decided? If not, you cannot measure adherence and you cannot debug a violation.
Most teams find the first list is shorter than expected and the third answer is no. That gap is the work.
What changes in the build loop
The design pays off in how the system changes over time, not on the first day.
A behavior change becomes a diff. Adjusting a threshold is a config change with a version and a review, not a prompt rewrite whose side effects are unknown until traffic hits it. Behavior you can change deliberately is behavior you can improve through an evaluation loop rather than by feel, since each change has a stated intent to measure against.
The right person can write the rule. A support lead can review a refund limit expressed as a value with a condition. Nobody outside the team can review the same limit buried mid-prompt, and they will not notice when it disappears in a rewrite.
Rules become test cases. Each declared rule already names a situation and a required behavior, so building the case set is transcription rather than invention. That set is what catches the regression when a model upgrade changes how instructions are followed.
Failures get attributable. When a rule exists, a failure resolves to one of three causes: the rule was missing, the rule was wrong, or the rule was not enforced. Without declared rules every failure looks like the model being bad at its job, which is where reliability work on agent systems tends to stall.
Where the approach breaks down
Not everything is a rule. An agent that should be helpful cannot be made helpful by a policy. Tone, explanation quality, and handling of situations nobody anticipated are model work, and the attempt to specify them exhaustively produces a system that says no to reasonable requests. Declare the constraints; leave the judgment.
Policy becomes the new prompt. Rules accumulate the same way prompt sentences do. Nobody deletes, conditions overlap, and after a year the policy set is as unreadable as the prompt it replaced. Rules need an owner, a review cadence, and a deletion path.
Declared is not enforced. Writing a rule in a config file that nothing reads is worse than the prompt version, because it looks like a control. Every rule needs a named enforcement point, and the ones with real consequences belong at a boundary the model cannot talk its way past.
Over-constraint hides as safety. Each rule narrows what the agent can do. Ten narrow rules can leave an agent unable to complete the task it exists for, and the resulting escalation volume gets blamed on model quality. Watch the rate at which policy blocks legitimate work, not only the rate at which it blocks violations.
Rules need the same context the agent has. A rule that depends on who the user is, what tier they are on, or what happened earlier in the session requires that state to be available at the decision point. Policy design and the way an agent’s context and tools are assembled are the same problem viewed from two directions.
FAQ
How is this different from writing better prompts?
Prompts and policies answer different questions. A prompt shapes how the model expresses itself and what it tries by default. A policy states what is permitted regardless of what the model decides. Better prompts raise the average case. Declared policy sets the floor, and the floor is what you get asked about after an incident. Keep both, and stop using the prompt to hold rules whose violation would be expensive.
What behavior should stay with the model?
Anything where the right answer depends on the specific situation: how to phrase a refusal, which of several reasonable approaches to try, when to ask a clarifying question. If you cannot write the rule without a paragraph of exceptions, it is judgment. Also leave the model anything where a wrong choice is cheap and recoverable, because a rule costs review, maintenance, and flexibility.
How do I check that the agent followed its policy?
Record a policy decision as part of the run, then evaluate against it. For deterministic rules the check is a comparison: the rule applied, the verdict was allow or deny, the agent’s action matched. For graded rules, a scored evaluation with the rule text as the rubric is the usual instrument, and its verdicts need validation against human review before the number means anything. Report results per rule, since one aggregate hides the rule that matters most.
Where do I start with an agent that is already running?
Read the system prompt and list every sentence that is actually a rule. For each, decide whether a violation is expensive. Move the expensive ones to a real enforcement point, one at a time, keeping the prompt sentence only where it helps the model cooperate. That ordering gets the consequential rules out of the prompt first and leaves the cosmetic ones where they already work.
