Building control foundations for trusted agentic AI

What should enterprises do about the advances in agentic AI? How can they build the necessary governance, security, identity, and control foundations needed to scale agentic AI responsibly and effectively?

Executive summary

The first article in this series argued that agentic AI is an operating-model shift rather than a tooling upgrade. This article sets out the operating model itself. It rests on a single principle – keeping probabilistic cognition under deterministic control – and four foundations: governance, security, identity, and control. It also adds a maturity path with evidence at each level, with mapping from controls to regulatory obligations, and the ownership model required to make it work. The aim is practical: to help organizations scale autonomy that they can trust, govern, and defend.

The principle: let the model reason, but never let it execute unchecked

Every durable agentic AI architecture rests on one idea: probabilistic cognition under deterministic control. A model is allowed to interpret, plan, and reason. It is not allowed to act unchecked. High-impact actions pass through typed interfaces, policy enforced as code, validation, approval where the impact warrants it, and rollback. Everything that follows elaborates on that principle across four foundations matching the questions leadership teams are now asking in governance, security, identity, and control.

Foundation one: Governance

Governance is the foundation that the others depend on, and it is largely a question of ownership. The first decision to make is to decide who owns the agentic control plane. In many organizations, responsibility is still distributed across IT, security, data, architecture, risk, and the core business. In practice, this means there is no single owner. That is a gap that leaders should close before agents reach production stage.

Practical governance includes a clear model and version strategy, with pinned versions, regression evaluations, and monitoring for behavior drift, since the same model name can behave differently over time. It includes a defined position on provider and deployer roles, documented human oversight, and an architecture for exit from providers. In our experience with regulated organizations, the exit question is often the most revealing. Teams can describe their models and use cases in detail, yet few can describe how they would leave a provider, or where their data, prompts, and logs would go if they did. An exit that cannot be demonstrated is not yet a true exit, and regulations such as DORA require financial entities to be able to demonstrate it.

This is also where digital sovereignty becomes practical rather than rhetorical. Sovereignty is the ability to keep the choice, cost, data, and audit trail under your control. The goal is not to choose between cloud and local AI, but to preserve strategic optionality: use frontier models where they create differentiated value; private or sovereign environments where sensitivity requires control; and portable governance patterns throughout.

Foundation two: Security

Agentic AI turns language into action, so security can no longer be treated purely as content moderation. The attack surface is at an operational level. Indirect prompt injection, demonstrated repeatedly by researchers, allows instructions to be hidden in a document, an email, or a web page that an agent then reads and acts upon, without any breached credentials. Also relevant here are poisoned memory, compromised tools, subagents exceeding their scope, and workflows that loop until they cause harm.

Security for agentic systems has three parts. The first affects the supply chain. Skills, tools, and the integration layer, whether implemented through MCP, a command-line tool, function calling, or an agent-to-agent protocol such as A2A, should be governed with allow-listing, scoped and short-lived credentials, sandboxing, egress control, and prompt revocation. A common pattern revealed in assessments is not a malicious component but a proof-of-concept integration running with a broad token and no egress control, later promoted to production with the same broad permissive scope.

The second part is exposure management matched to AI speed. Google’s Big Sleep cybersecurity project shows the defensive side necessary to meet AI’s acceleration effect on enterprises: cybersecurity AI agents can be used to discover exploitable vulnerabilities before release, or before broad exploitation. Attackers have the same capability, which compresses the time available between discovery, proof, and exploitation.

Monthly patch cycles struggle to keep pace, so prioritization has to shift from assessment based on raw severity alone towards exposure. A “gravity” model is one practical way to do this, weighting a finding by exposure, privilege path, and blast radius rather than score alone:

The third part of this foundation is detection and response designed for autonomous actors: monitoring agent behavior, anomaly detection, and having the ability to reconstruct what an agent did, when, and on whose authority.

Foundation three: Identity

Every agent requires credentials, making it a non-human identity. Recent identity-security research indicates the scale of the issue: machine identities now far outnumber human identities, and AI agents are part of that population. At the same time, many organizations still lack mature controls for ownership, scope, monitoring, and revocation.

Palo Alto Networks’ 2026 research puts the ratio at 109 machine identities for every human identity. Excessive access remains a persistent problem across both human and machine environments.

Many agents today are provisioned poorly: they borrow human access, lack clear ownership, hold more privilege than they use, and have no expiry. The response is to treat agents as first-class identities within a Zero Trust model. Each agent should have a unique name, a defined owner, and a stated purpose. It should operate with least privilege, use scoped and short-lived credentials, and rely on just-in-time access. Controls must include full logging, continuous evaluation against policy, and a revocation path that works under pressure.

The starting point is usually an inventory, since in many assessments organizations often cannot produce a current list of their non-human identities, let alone the owner and expiry for each. You cannot govern what you cannot name.

Foundation four: Control

Control is where the principle above becomes operational: it is the rails and guardrails between an agent’s reasoning and its actions. Deterministic rails ensure that agents’ high-impact execution passes through typed interfaces, policy as code, and validation. Guardrails, in turn, apply across reasoning, tools, execution, memory, skills, delegation, identity, economics, and browsing.

The practical distinction is reversibility. Low-impact, reversible actions may be suitable for human-in-the-loop supervision, where the agent operates under monitoring with the option for intervention. High-impact or irreversible actions, such as privilege changes, production deployment, regulated-data movement, payment initiation, or external commitments, require stronger controls and, in many cases, explicit human approval before execution.

The 2012 Knight Capital incident, in which financial trading automation caused a $440 million loss in minutes before it could be halted, illustrates why a reliable stop control and clear intervention path matter, even for deterministic systems. The same failure mode is more consequential for a reasoning, tool-using agent. Control also includes economic guardrails, because agents loop, retry, and call tools, and cost can accumulate before a person notices.

A maturity model for trusted autonomy

A simple maturity path helps an organization evaluate its level of trust and plan the next steps for improvement. Evidence matters more than intent, so each level of autonomy is defined by what can be shown.

• Level 0 – Ungoverned: agentic pilots in business units, with no central ownership, no inventory, and human-borrowed credentials.
• Level 1 – Visible: an agent inventory, named owners, and basic logging, with high-impact actions requiring approval.
• Level 2 – Governed: scoped agent identities, deterministic rails on high-impact actions, and tool allow-listing, supported by exposure-based vulnerability prioritization.
• Level 3 – Resilient:: tested stop controls, anomaly detection for agent behavior, and regression evaluations, mapped to AI Act, DORA, and NIS2 obligations.
• Level 4 – Trusted autonomy at scale: demonstrable exit plan, independent verification, and regulator-ready evidence of controls, with multi-provider portability.

Many organizations today sit between Level 0 and Level 1. There is measurable value in moving up this scale deliberately.

Mapping the foundations to regulation

The four foundations are not only good practice. Each supports obligations that are already in force or imminent. The mapping below is indicative only and should be confirmed by legal specialists for the relevant sector and jurisdiction.

Agentic AI – who owns it?

Trusted autonomy is a shared responsibility with a single accountable owner. The control plane benefits from an executive owner, which is increasingly a partnership between a company’s CISO and the head of AI or data. The security pillar is accountable for identity and guardrails, architecture for the rails and integration standards, data for retrieval and provenance, withrisk and legal taking on regulatory mapping. Use cases and their impact classification are held by the business. The pattern for failure is usually the inverse: a program that touches governance, security, and identity but reports to none of them.

From experimentation to trusted autonomy

The agentic AI phase rewards a different capability from the generative phase. It is less about experimenting quickly and more about scaling autonomy  based on sound principles and foundations. That is the practical meaning of digital sovereignty., and it is the journey from agentic experiments to trusted autonomy at enterprise scale. An essential first step is small but clarifying: name the agents, assign their owners, and put the organization on the maturity path. The rest builds from there.

Questions executives should ask now

• Do we have an inventory of agents and non-human identities, with named owners?
• Can we classify agent actions by impact and reversibility of process?
• Do high-impact or irreversible actions pass through deterministic controls?
• Can we exit critical AI providers and retain the audit trail?
• Do AI, cybersecurity, data, legal, risk, and business teams share one operating model?

How Capgemini helps

Capgemini helps organizations move from agentic experimentation to trusted autonomy. We do this by first assessing maturity, defining the control-plane operating model, securing agent identities and integrations, and mapping controls to regulatory obligations. This is our basis for designing scalable architectures that balance innovation, sovereignty, and resilience.