Share


As enterprises move into the agentic era, AI has quietly changed jobs. A year ago, most deployments drafted, summarized, and recommended. Today they open tickets, adjust records, release payments, and hand work to other agents. The person who used to sit between the recommendation and the action has stepped out of the middle. Forrester expects an Agentic AI deployment to cause a publicly disclosed breach in 2026, serious enough to cost people their jobs. A system that owns an outcome is a system that acts, and action needs different governance than advice. Nowhere does that difference matter more than in how enterprises decide what their AI is allowed to do.

The growing gap in AI governance

Ask a business unit how many agents it runs, who owns each one, and what systems they can reach. Very few can answer with confidence. Capable teams built these agents to solve real problems, one project at a time, and the result is a population nobody governs.

Security leaders see the exposure clearly. In a Dark Reading poll, 48 percent of cybersecurity professionals named Agentic AI the top attack vector for 2026, ahead of deepfakes and every other category. The cost side is just as concrete: IBM’s 2025 breach research found that incidents involving unsanctioned “shadow AI” average $4.63 million, roughly $670,000 more than a typical breach.

In July 2026, the exposure moved from survey data to the news cycle. OpenAI disclosed that advanced agents under evaluation found weaknesses in their test sandbox, escaped its limits, and went on to breach Hugging Face, one of the world’s largest hubs for sharing AI models, reaching some of its internal systems before the incident was contained. Hugging Face closed the vulnerabilities, rebuilt the affected systems, and summed up the moment plainly: “autonomous, AI-driven offensive tooling is no longer theoretical.” A sandbox built to bound the agents was itself the first system they defeated.

The symptoms follow a pattern. An agent takes a step that was technically permitted but commercially wrong, and nobody can reconstruct which decision led to it. A service account created for a pilot becomes the identity a sub-agent quietly uses in production. Permissions granted for one workflow get inherited by another that nobody reviewed. Speed compounds all of it because agents decide in milliseconds and at volume, so human approval queues turn into bottlenecks or rubber-stamp processes.

The controls already in place were built for models: accuracy testing, bias review, explainability documentation. That work still matters, but none of it tells you whether a specific action was authorized, bounded, reversible, or observable.

Machine agency, reimagined as the object of governance

Model governance asks whether a system knows well. It examines accuracy, fairness, and explainability, and those were the right questions when a human stood between the output and any consequence. Agent governance asks whether a system acts well. Was the action authorized? Was it bounded in advance? Is it reversible, observable, and attributable to someone who can answer for it?

This is why governing machine agency belongs alongside NIST AI RMF, ISO/IEC 42001, and the EU AI Act rather than inside them. Those frameworks are still necessary and in force, but each rests on assumptions that autonomous action breaks: that a human reviews consequential decisions, that permissions stay static, that a single deployer is accountable, and that offline testing predicts live behavior. Regulators have started to close the distance. In January 2026, Singapore’s IMDA launched the world’s first Model AI Governance Framework built specifically for agentic AI, structured around bounding risk upfront, meaningful human accountability, technical controls, and end-user responsibility. The security community made the same turn: OWASP’s Top 10 for Agentic Applications, published in December 2025 and peer-reviewed by more than 100 practitioners, treats agents as principals with goals, tools, and delegated authority, a threat model built around identity and action rather than the network perimeter.

Treating agency as the governed object changes what an enterprise builds. Every agent in production should be able to answer five questions at any moment. 

  • Who is watching, and can they stop it? 
  • Who allowed this, and on whose authority? 
  • What could go wrong, and how is it bounded in advance? 
  • Who answers for what it did, and with what evidence? 
  • What detects and halts harm during execution? 

Those five questions become five pillars: human oversight, authority and delegation, risk management, accountability, and runtime controls.

Figure 2 Five questions five pillars 1024x499 1

Agency is delegated authority, which makes this the oldest problem in governance wearing new clothing. One rule governs it: an agent never holds more authority than the person or role it acts for, and a sub-agent never inherits more than its principal. Making that real means encoding decision rights at the workflow and tool layer instead of in prompts, where limits are easily bypassed. Anthropic’s own engineering guidance points builders the same way, favoring simple, composable workflow patterns over unconstrained autonomy. Monetary ceilings, record counts, and rate limits get enforced deterministically. Proposal, approval, and execution sit with distinct agents or roles. Every delegation is scoped, time-bound, and recorded at each hop.

Two capabilities carry most of the operational weight. The first is an enterprise agent registry, a single authoritative record of each agent’s identity, owner, authority, risk tier, and dependencies. The World Economic Forum reaches a similar conclusion in its work on agent evaluation, proposing an “agent card” that documents each agent’s capabilities and operating context before it is onboarded, much like a resume for a new employee. Without a registry, an enterprise governs scattered projects instead of a managed population. The second capability is runtime control: policy checked against each action before it executes, the ability to halt an individual agent or the whole system, and every decision landing in structured, immutable logs.

Proven where the stakes are highest

A bank deploying an exceptions-handling agent in payment operations sets its monetary ceiling in the workflow layer rather than the prompt, and separates the agent that proposes a fix from the one that executes it. Routine volume clears faster, and irreversible cases still reach a human.

A health insurer automating prior authorization triage routes clear approvals autonomously and sends every denial path to a clinician. Immutable decision logs turn regulatory review into a retrieval task instead of an investigation. A manufacturer running procurement agents across its supplier base caps order values and monitors agent-to-agent messages, so a single bad demand signal cannot cascade through the chain before anyone notices. In each case, the controls are what allow the deployment to grow.

Figure 3 Representative deployments in regulated industries 1024x486 1

From frameworks to systems that answer for themselves

Autonomy earns trust the way people do: by working inside known limits, in the open, with someone answerable for the result. As Altimetrik’s AI-First enterprise vision makes clear, the future belongs to systems that operate continuously, own outcomes, and earn trust through transparency and control. Enterprises that build this capability early will not run slower. They will be the ones able to grant their agents more authority, sooner, with a defensible reason to trust them, while competitors are still explaining their first public incident.

Dattatray Indore
About the Author
Dattatray Indore
Senior Manager, Product and Industry Practice, Altimetrik
Dattatray Indore is Senior Manager, Product and Industry Practice at Altimetrik, where he works at the intersection of enterprise payments, applied AI, and governance for regulated industries. An independent AI governance researcher and the author of the SSRN paper Governing Machine Agency, he argues that autonomous systems require a governance discipline distinct from traditional AI model governance, built around delegated authority, runtime control, and engineered accountability.

References

  1. “Predictions 2026: Cybersecurity and Risk Leaders Grapple with New Tech and Geopolitical Threats,” Forrester, October 2025, https://www.forrester.com/blogs/predictions-2026-cybersecurity-and-risk 

  2. “2026: The Year Agentic AI Becomes the Attack-Surface Poster Child,” Dark Reading, 2026, https://www.darkreading.com/threat-intelligence/2026-agentic-ai-attack-surface-poster-child 

  3. “Securing AI Agents: The Defining Cybersecurity Challenge of 2026” (citing IBM Cost of a Data Breach Report 2025), Bessemer Venture Partners, 2026,
    https://www.bvp.com/atlas/securing-ai-agents-the-defining-cybersecurity-challenge-of-2026 

  4. “OpenAI Says Its AI Went Rogue and Launched ‘Unprecedented’ Cyber-Attack,” BBC News, July 22, 2026, https://www.bbc.com/news/articles/c3ek3gvdnj3o 

  5. “Artificial Intelligence Risk Management Framework (AI RMF 1.0),” National Institute of Standards and Technology, January 2023,
    https://www.nist.gov/itl/ai-risk-management-framework
     
  6. “ISO/IEC 42001:2023, Information Technology, Artificial Intelligence, Management System,” International Organization for Standardization, December 2023,
    https://www.iso.org/standard/81230.html 

  7. “Regulation (EU) 2024/1689 (Artificial Intelligence Act),” European Union, July 2024,
    https://eur-lex.europa.eu/eli/reg/2024/1689/oj 

  8. “Singapore Launches New Model AI Governance Framework for Agentic AI,” Infocomm Media Development Authority, Singapore, January 22, 2026,
    https://www.imda.gov.sg/resources/press-releases-factsheets-and-speeches/press-releases/2026/new-model-ai-governance-framework-for-agentic-ai 

  9. “OWASP Top 10 for Agentic Applications for 2026,” OWASP Gen AI Security Project, December 2025, https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/ 

  10.  “Building Effective Agents,” Anthropic, December 2024,
    https://www.anthropic.com/engineering/building-effective-agents 

  11. “AI Agents in Action: Foundations for Evaluation and Governance,” World Economic Forum with Capgemini, November 2025,
    https://www.weforum.org/publications/ai-agents-in-action-foundations-for-evaluation-and-governance/ 

  12. Dattatray Indore, “Governing Machine Agency: A Governance Discipline and Operating Model for Autonomous, Multi-Agent, and Decision-Taking AI Systems in Regulated Enterprises,” SSRN, July 3, 2026 (doi: 10.2139/ssrn.7052199),
    https://papers.ssrn.com/sol3/papers.cfm?abstract_id=7052199 
More Industry Insights

Harnessing Altimetrik’s Expertise

Contact Us

We'd love to hear from you.
Contact Us