Enterprises are embracing agentic AI at an accelerated pace, often faster than their governance models can support. The drive for efficiency and innovation may push experimental agents into production, creating new and underappreciated risks for IT operations and cybersecurity teams.
The challenge is not just to manage individual agents, but to govern a sprawling ecosystem of agents that interact, delegate, and execute decisions at machine speed across the enterprise. When agents launch a series of automated workflows, small misalignments or misconfigurations can compound quickly, especially as agents plan new approaches, create and delegate to subordinate agents, access sensitive data, or reconfigure systems faster than humans can track, much less manage.
Last November, McKinsey reported that 23% of organizations are already scaling an agentic AI system somewhere in their enterprise, and another 39% are experimenting with agents. In AI-adoption-years, November 2025 feels like a generation ago, and adoption in mid-2026 is most certainly higher. The release of Anthropic’s Opus 4.6 model in February 2026 seemed to light a fuse across the industry and correlates with Anthropic’s own March 2026 reporting showing that specific automation-focused API workflow category usage doubled since November 2025, and indicates an increase toward automation-driven, agentic work via Claude Code and the Claude API (and less of an increase in the chatbot-style interactions with Claude.ai). This pace will also widen governance gaps that become harder to detect and more costly to correct.
In March, the source code for Claude Code (the command-line front-end for interacting with Claude) was accidentally published by Anthropic. While it did not release the crown jewels of Anthropic’s training methods or model weights, it did give researchers more of a view into one layer of Anthropic’s protections. The guidance is, at least on the client side, “defense by vibes” (a phrase my own instance of Claude referenced when “we” analyzed the source map leak together). In the April 7 Mythos Preview system card, Anthropic states, “Claude Mythos Preview is, on essentially every dimension we can measure, the best-aligned model that we have released to date by a significant margin. Even so, we believe that it likely poses the greatest alignment-related risk of any model we have released to date.” It’s powerful, but it demands guardrails. They go on to explain that these capabilities that make the model so powerful are also what they expect will increase risk. As in mountaineering, a more experienced guide is necessary to take you on more dangerous climbs, but that doesn’t make it less dangerous to do. Even when better aligned, the actions taken may still be incredibly risky.
The adoption of highly flexible agents requires governance models that match their autonomy, judgment, and scale.
While AI instruction sets and preferences may become part of a comprehensive defense-in-depth strategy, it’s a hard pill to swallow for most security purists. We’re accustomed to leveraging formal, measurable, precise technical controls, but teaching AI agents to play by the rules includes things like, “If you suspect malicious behavior, flag it,” in a preferences file or instruction set. This may help prevent some breakdowns, but it is just a guideline, not a rule. The adoption of these highly flexible agents requires governance models that match their autonomy, judgment, and scale. As organizations rush from experimentation to production, IT and security leaders must reassess not just what agents do, but how systems of agents are monitored, constrained, and held accountable.
The risks of agentic AI
For three and a half years, organizations have experimented with generative AI and are still coming to grips with the risks that may accompany its benefits. Collectively, we’re learning a lot about memory and context management, safeguards and hooks, and how to pressure-test hallucinatory behavior. These challenges are mostly limited to bad information in a conversation with AI, but agentic workflows can rapidly amplify damage. GenAI is frequently associated with chatbot-style interactions and prompt-response cycles. Agentic AI goes a step further: It takes independent action, and therein lies a greater danger. AI agents can execute at scale, moving faster and more broadly than human oversight can track. They can also compound bad decisions across a workflow without the corrective force of reflective judgment.
What was previously working theory is rapidly becoming reality. The Mythos Preview System Card documented the power of an early version of the Mythos model to creatively reason its way to successful exploitation of enterprise software. The industry saw flashes of what an agentic adversarial ecosystem might bring, and how agentic tooling can take unexpected paths to achieve its objectives. Anthropic applied training interventions to substantially improve these behaviors in the final version of Mythos Preview, and these findings will inform the safety work in future models. The nature of AI is that even best-in-class alignment doesn’t guarantee perfect rule-following, and more powerful models can have more impact with unexpected actions they take.
The price of agentic sprawl extends beyond the considerable security and compliance risks. Individual agents may make conflicting decisions for different work teams or business departments, causing uncertainty or cycles of conflict. Redundant workloads or runaway behaviors can cause infrastructure costs to spike or availability to plummet. Or imagine an agent entrusted to accelerate patch compliance that does so by forcing immediate restarts across the entire network, corrupting in-flight processes and creating widespread service disruption.
An over-eager agent decided that the most efficient way to clean up a file directory was to delete anything older than six months.
Imagine another tasked with tidying up email inboxes, but then archives emails still requiring critical action or that demand special handling for compliance purposes. I write from personal experience: An over-eager agent decided that the most efficient way to clean up a file directory was to delete anything older than six months. When my pre-existing hooks stopped it from running a command to delete, it found a command I had not explicitly told it to avoid and used that instead. It was a valuable lesson for me and the AI that day (and don’t worry, I had backups).
Without proper governance and human-in-the-loop (at the right place in the loop), actions that seem to have modest scope and benign intent can cause harm across the enterprise. Six significant risks of putting agentic AI into production without strong guardrails are:
- Sprawl: Agents can quickly multiply, fragmenting work and creating overlap across teams.
- Configuration creep: As prompts, preferences, and configurations drift, the activity and impact of agents can change, resulting in them moving beyond their original intent.
- Brittle operational infrastructure: Ungoverned agents can make systems fragile, such that small changes trigger unpredictable failures.
- Runaway costs: Agentic AI can use a lot more compute than GenAI chatbots, and a swarm of undermanaged agents running 24/7 can be shockingly expensive.
- Slowed incident response: Without oversight, it becomes difficult to trace which agent acted, why it acted, or how far its impact has spread.
- Mistrust in autonomous systems: Unpredictable agent behavior erodes confidence in automation and slows adoption.
Governance helps keep agentic AI on track by setting clear boundaries, validating behavior against defined intent, and making agent actions visible and accountable. Through sandboxing, testing environments, human‑in‑the‑loop oversight and transparent decision-making, organizations can prevent uncontrolled growth, catch risky behaviors early, and build trust in autonomous systems before issues reach production.
Elements of agentic governance
The goal of agent governance is to ensure safe actions and outputs that are aligned with security practices, regulatory obligations, privacy practices, and business goals. Achieving these goals demands tight collaboration between IT and security teams. I’d argue, in fact, that these teams can no longer afford to think of themselves as separate teams at all. Their workflows and the pace at which they work will be so tightly intertwined that they will need to work in lock-step with each other and their agentic tech. Governance and collaborative strategy must be established early and evolve alongside AI capabilities.
Human-driven policies of governance must be in place to avoid or contain an unnecessary explosion of agents.
We can divide concerns about the proliferation of agents into two categories: human and technical. Humans will unintentionally engineer agentic sprawl when individuals and teams spin up new agents at will to handle focused tasks, perhaps without consideration of overall best practices about how and when to create, operate, and retire an agent. These human-driven policies of governance must be in place to avoid or contain an unnecessary explosion of agents. Some of the technical considerations in the next section also touch on this risk.
Turning to the technical aspects of governing enterprise AI agents, we begin with the basics:
- Test agents extensively before deployment. This cannot be overstated.
- Define ownership and accountability for each AI agent.
- Limit agent permissions using least-privilege access controls.
- Log and monitor agentic activity to enable auditing and troubleshooting.
- Establish policies for acceptable use and regulatory compliance.
- Keep humans in the loop for high-risk or irreversible actions. AI is still in its early era, and some tasks are better suited to early adoption than others.
These are table stakes; cybersecurity and IT operations teams should go a few steps further:
- Treat AI agents as a new class of actor. Agents are not typical deterministic software services, nor are they accountable or expected to have ethical and contextual judgment in the way human employees would. Enterprises should define a separate classification for agents that takes their autonomy and probabilistic behavior into account and sets appropriate policies, review processes and escalation paths. The agent is an identity and an application and a behavior to monitor all in one.
- Govern an agent's intent, not just its access. Controlling APIs is a key step in governing AI agents, but it's not enough. The enterprise must define what an agent is trying to achieve and what the limitations on its actions are. This includes its ability to create sub-agents or to work with other agentic systems that may have higher permissions, conflicting goals or different limits.
- Observe behavior, not just outcomes. It is important to assess not only an agent’s outputs, but its decisions in context. If you don’t know why and how an agent moved from prompt through decision options to final action, you don't really know whether it is functioning correctly. This level of traceability is also vital for compliance and security incident investigations. This element is one of the most challenging to get right, and does not have a one-size-fits-all answer today.
- Limit their independence. Beyond least privilege access controls, hold agents to a standard of certainty for taking action. Require agents to bring a human into the loop when confidence in a proposed action falls below an acceptable threshold. Overconfidence is a hallmark of LLM outputs, and there can be considerable risks when an AI agent essentially bluffs through uncertainty rather than escalating for support.
- Expect error and design for containment. Agents will eventually act in an unexpected way. Organizations should limit the scope of systems that an agent can affect and cap the frequency with which it can take action to allow organizations to disable an agent in a timely manner, with limited damage.
- Regularly assess permissions and objectives. Agents are adaptive systems working in ever-changing environments. As prompts, models and organizational needs change, outdated permissions and objectives may create uncertainty or risk.
These principles should help to keep any given agent in line. But to manage the further risk of agentic sprawl, there are additional considerations that treat agents as a system.
Four factors for agentic crowd control
In isolation, a poorly governed agent poses risks to the organization. When the number of agents in the enterprise grows exponentially, so does the possible risk. As the enterprise brings more agents into production (and those agents perhaps spawn sub-agents to complete their tasks), governance must take on a new dimension. You’re not just governing individual tools, but a fast-moving, complex ecosystem of probabilistic machine actors. Key considerations at this level include:
- Centralized inventory and agent registry. As with endpoints, you can’t manage the agents you can’t see, so establishing a centralized inventory is the first task. With this accomplished, consider not merely individual agents, but the entire portfolio. As a whole, does the system of agents reflect the enterprise’s mission and goals? Do the agents collectively pose risks that may not be apparent at the individual level?
- Continuous observability and comprehensive governance. To govern a system of agents, you need visibility into the entire system. A consolidated view reveals how agents interact, where authority overlaps, and how risk accumulates across the agentic system. Centralized monitoring and governance allow organizations to apply policies consistently across the agentic system, detect drift over time, and intervene quickly when agent behavior deviates from enterprise intent.
- Lifecycle management. Agents forgotten over time, or replaced with better tools but never decommissioned, may continue to act, use resources, or become an unrecognized attack surface. Enterprises need clear plans to identify and retire agents that are redundant or that no longer serve the business.
- Resource usage tracking and cost management. The cumulative effect of agents on enterprise resources has to be taken into consideration. Any given agent may be within parameters and providing efficiencies, but it’s important to understand overall resource usage and to have insight into where it is exceeding parameters.
While the challenges of a sprawling agentic ecosystem manifest over time, well after the initial agents enter production, system-level thinking about agentic governance should be present at the start. Before the first agent rolls into production, organizations should consider how to manage and secure a broader ecosystem of interacting agents.
Governance makes the agentic enterprise scalable
Today, we are attempting to force-fit probabilistic systems into binary technical controls of old. There is a balance to be struck at the intersection of adopting innovative technology, changing the conversation about what technical controls do (and do not) adapt to, and translating governance for the modern era. The organizations that apply a new lens to what good governance means will enable digital transformation and protect resilience in the process. Those organizations will pull away from the pack in productivity and innovation. Doing it right is not about limiting AI; it’s about ensuring that autonomous systems consistently act in service of intended business outcomes.
While the foundations of governance are familiar to security and operations teams, agentic AI fundamentally changes the scale, pace, and shape of the challenge. Governance must now account for evolving systems of agents that act and interact beyond direct human oversight, and sometimes those that operate on “defense by vibes,” which may be an uncomfortable new frontier for those who are seasoned professionals in the field. In that sense, organizations must return to first principles—not to recreate old controls, but to rethink governance from the ground up for an era of autonomous decision‑making. Done well, that governance will prevent agentic sprawl from becoming destructive chaos and ensure that speed and scale remain assets rather than liabilities.