AI agent permissions determine what an agent is allowed to do on someone’s behalf. As agents get more powerful, we’re running into a basic problem: a model can’t reliably verify those permissions just because someone claims to have them in a conversation.
The problem agents have with permission goes both ways: sometimes they steadfastly refuse to do something you’re authorized to do, but looks dangerous. Other times, they blithely accept a persuasively-worded false authority to do dangerous things. Even worse, sometimes they accept persuasive words you didn’t even write.
Permission is one part of a huge bucket of things referred to by the AI community as “safety,” but if I tried to write about all of safety at once this post would never end. So let’s focus on permission: how things generally work right now, why that’s not good enough, and where things are going. (For the broader guardrails picture, see agent guardrails vs evals.)
Build better agents with Arize
Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.
Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.
How Claude Code detects dangerous commands
One of the biggest rules of computer security is that your defaults should be safe. Here, the model makers have been making some strides: Claude Code’s auto mode, made the default for new sessions on Pro, Max, and Team plans in August 2026, runs every tool call through a classifier built to block actions that are irreversible, destructive, or aimed outside your environment. In Anthropic’s study of 1,053 paid testers, human review caught just 13.6% of planted dangerous commands while auto mode caught 89%, roughly 6.5× the human rate.
Good defaults are great! Auto mode is definitely safer than “allow everything” and also probably safer than “ask every time”, since we probably all remember from experience getting tired or bored of Claude Code constantly asking us things and just hitting “allow” to everything without reading. But there’s a problem with making the defaults avoid destructive things, which is that sometimes you want to be destructive on purpose.
Why agents refuse tasks you’re authorized to do
Sometimes, you really do want to delete your home directory. Sometimes, you want to drop the database. Sometimes, especially if you work in security, running exploits against a remote machine is exactly what the agent should be doing. The problem is, the agent has no way of knowing when this is true, so the defaults work against you and make life frustrating and the agent less useful.
It can even be actively harmful: when HuggingFace was being hacked (by OpenAI, it turned out) they tried to use Claude to reverse-engineer what the attacker was doing, and Claude refused. They eventually had to turn to Chinese models with fewer safeguards to make progress. That’s a broken permission model making a company demonstrably less safe. These defaults don’t always work.
And that’s not an isolated incident. On Artificial Analysis’s Cyber Index, a benchmark of cyber defense tasks, several frontier models decline tasks covering 32% to 38% of the index on safety grounds. The models score zero at those tasks, not because they’re incapable of doing them, but because they refuse to believe the person doing the asking is who they say they are, namely a defender of cyber-attacks.
The agents need to know who’s asking, and they are singularly bad at doing that.
Why agents accept false claims of authorization
In late 2025, Anthropic reported that it had disrupted a Chinese state-backed group, GTG-1002, that ran a cyber-espionage campaign using Claude Code. The operators got past its safeguards by telling it a story: that it was an employee of a legitimate cybersecurity firm, being used in defensive testing. On that footing the agent ran 80% to 90% of the attack, with the humans stepping in only at a handful of decision points. In Claude’s defense, how was it supposed to know it was being lied to?
A company called LayerX demonstrated another way to do this: they added a claimed pentest authorization into a project’s CLAUDE.md file, and in their writeup Claude explicitly cited the file as its authorization before helping with login bypass and database dumping.
How easy is this to do? Way too easy. In the Open Agent Passport adversarial testbed, researchers reported a 74.6% social-engineering success rate when the model operated under a permissive policy.
The fundamental problem with a model’s idea of what permissions you have is that it has no source of truth other than what you tell it, and you can tell it whatever you want. This isn’t really the model’s fault, and it’s not going to be fixed by smarter models. But that’s not even the biggest flaw in the system.
How prompt injection exploits agent permissions
Prompt injection is an attack that tries to redirect an AI agent through instructions in untrusted content. Those instructions can arrive in a user message, a retrieved document, a web page, or a tool response. The permissions problem gets worse when the agent mistakes an attacker’s words for a request or authorization from the user.
Anthropic’s guidance for Opus 5.5 tells developers to wrap pasted-in text in tagged blocks to distinguish the user’s own instructions from text they pasted from elsewhere. But in the same instructions it makes it clear that this can be faked, and can’t be the only thing you rely on.
So “safe” defaults aren’t always correct, the agent has no way of knowing whether you should be able to override those defaults because it has no idea who you are, and it’s not even sure whether it’s you doing the talking. This is a nightmarishly broken permissions model, and everyone has been pretty vocal about saying so. So the model makers are starting to make some progress on fixing it.
How AI labs verify authorization outside the chat
In early October 2026, Anthropic expanded its Cyber Verification Program (CVP), a program that allows legitimate cyber-defense professionals access to models with advanced abilities in offensive techniques and reduced guardrails about using them.
The permission step happens outside the chat through verified account access. In Anthropic’s CyScenarioBench evaluation, Claude Opus 5.5 was tested five times on each of 10 offensive scenarios. Without CVP access, all 50 trials were blocked at the first prompt. With Red Team Access, no blocks occurred and the model completed 34 of the 50 trials.
OpenAI has an equivalent program, called Daybreak, with similar results: the trusted-access version scores 32 points higher than the public one on Artificial Analysis’ cyber index.
But this is a specific solution to the problem of cyber-defense, not a general fix for the way models handle permissions. Luckily, there are some best practices you can put into place in the agents you build that can help you not get constantly owned by people who ask to do evil things in a persuasive way.
How to build a permission model for AI agents
The base principle is simple: a claim of permission in the chat is not proof of permission. Enforce authorization in the agent harness, the application code that controls tool execution, before allowing an action to run.
Verified permission can come from your role in an organization, ownership of a resource, or a delegation of authority your application has validated. The harness should check whether that permission covers the specific action and resource. That check must run even when the agent claims it already has approval.
While you’re at that, there’s a few other best practices to follow: log at the tool level where permission came from, because the agent can be fooled into thinking it already called the tool, or that it doesn’t need to, or any number of other tricks. It’s important to log it somewhere the agent can’t reach, because agents have got good at covering their tracks, and a capable attacker will try to do that too.
Which brings me to my inevitable plug: this is why you need trace-level observability of your AI applications (see also AI observability tools for agents). With the relevant tool calls instrumented using OpenInference conventions, Arize AX lets you inspect their inputs, outputs, and surrounding context. Record the permission check and its result alongside the action so you can investigate what was authorized and what actually ran.
Agents are getting handed real authority to do useful (and potentially dangerous) things way faster than most of us expected. Giving them a real way to know who’s asking isn’t a research problem, it’s a critical production question that applies right now, so it’s time to give them one.
