AI agents are being given more responsibility inside business environments. They can interact with applications, access data, call APIs, use tools, and complete multistep tasks with limited human involvement. Every additional capability creates another decision an organization must make about what the agent should be allowed to do.

Recent research shows why those decisions need to be enforced through security architecture.

In his Dark Reading article, “AI Model Rules Are Not Security Controls,” Jacob Krell examines an OpenAI incident in which agents found an unintended communication channel and ultimately participated in activity that reached Hugging Face production systems. According to Krell, some agents demonstrated awareness that their actions fell outside the intended scope, yet continued pursuing them.

For organizations deploying agentic AI, the incident raises an important question: What actually prevents an AI agent from taking an unauthorized action?

Rules and Security Controls Serve Different Purposes

Organizations can give an AI agent detailed instructions about what it may access, which tools it may use, and where its authority ends. Model safeguards, system prompts, classifiers, and monitoring can all help shape behavior.

Security teams still need enforceable boundaries around those instructions.

Consider an agent that has been instructed not to communicate with systems outside an approved environment. The instruction provides behavioral guidance. Restricting network access so those systems cannot be reached provides a technical control.

Krell makes this distinction particularly clear. An agent may recognize that an action conflicts with its instructions and still determine that the action helps accomplish its objective.

For businesses, that means AI security architecture should account for what an agent can do as carefully as what it has been told to do.

Limit the Agent’s Ability to Act

Fortunately, many of the principles needed to secure agentic AI are already familiar to cybersecurity teams.

Least privilege can limit an agent to the permissions required for its assigned task. Network segmentation can restrict which systems it can reach. Credential controls can reduce unnecessary access to secrets, tokens, and APIs. Tool restrictions can prevent agents from invoking capabilities that fall outside their intended role.

Organizations should also identify higher-risk actions that require explicit human approval. Accessing sensitive data, changing system configurations, executing privileged commands, or attempting to expand an assigned scope are examples where deterministic controls and human oversight may be appropriate.

The objective is to limit the potential impact when an agent behaves unexpectedly.

Test What the Agent Can Actually Reach

AI security assessments should examine more than the model itself.

Security teams need to understand the environment surrounding the agent. What systems can it access? Which credentials are available? Can it communicate through unintended channels? What happens when it attempts a prohibited action? Will monitoring recognize the behavior, and when will a human be alerted?

The OpenAI example described by Krell demonstrates the importance of this approach. The agents reportedly used writable infrastructure as a communication mechanism, showing how capabilities that appear routine can create unintended paths through an environment.

Testing those paths can expose weaknesses before they create a larger problem. Carson & SAINT’s penetration testing services similarly focus on validating how security controls perform when confronted with real attack techniques rather than assuming configuration alone provides sufficient protection.

Include AI Agents in Your Risk Assessments

As organizations adopt agentic AI, these systems should become part of the broader Cyber Risk Management process.

Security leaders should know which agents are operating, what business processes they influence, which systems and data they can access, and what controls limit their authority. They should also understand the potential business impact if an agent takes an unexpected action.

That visibility allows organizations to apply familiar risk-management principles to a rapidly developing technology.

Know What Stops the Agent

Agentic AI can create meaningful business value, but greater autonomy requires stronger boundaries.

Instructions and model-level safeguards remain valuable parts of an AI security strategy. Organizations also need technical controls that behave predictably when an agent reaches the limits of its authority.

Before giving an AI agent access to sensitive systems or business processes, security leaders should be able to identify exactly what prevents it from going further than intended.

If that boundary depends primarily on the agent choosing to obey its instructions, agentic AI security controls still need work.

For assistance on Cyber Risk Management and security controls, Contact us here.