5 Questions To Ask Before AI Becomes One Of The Most Privileged Systems In Your Organisation

Over the last year we’ve spent a considerable amount of time sitting with leadership teams as they work through the practical realities of deploying AI. The conversations are often framed around familiar questions. Which model should we choose? Should we self-host? How do we satisfy our regulator? What are the implications of the CLOUD Act? Can we safely deploy agents? Each question is reasonable on its own. Collectively, however, they often obscure the more fundamental issue.

Before allowing AI to become one of the most privileged systems inside your organisation, we believe every leadership team should already be able to answer five questions:

• Who decides what the AI can see?
• Who decides what the AI can say?
• Who decides what the AI can do?
• Who decides when the AI must stop?
• Who ultimately remains in command?

Over the last year we’ve found ourselves returning to these questions in almost every sovereign AI deployment. Interestingly, almost none of them are really about AI. They’re questions about governance, authority and organisational control that AI simply forces into the open.

The interesting thing about enterprise AI is that almost nobody consciously decides to build one of the most privileged systems inside their organisation. They simply set out to build a useful one.

It begins by answering questions against a handful of documents. Someone then asks whether it can search Confluence. Another team wants access to Jira. Engineering connects GitHub. Legal asks whether it can reference contracts. Customer Success wants it to understand support history. Finance would like it to retrieve commercial information before drafting responses. Eventually somebody asks a perfectly sensible question: if the AI already understands what needs to happen, why shouldn’t it simply do it?

None of these decisions are individually remarkable. In fact, we’d probably encourage most of them. The value of enterprise AI comes from context, and context comes from connecting systems that have historically operated in isolation. What surprised us wasn’t the direction of travel. It was where that journey consistently ended.

Useful AI systems become extraordinarily privileged.

Not because organisations deliberately design them that way, but because usefulness naturally accumulates access. Unlike a CRM or an accounting platform, AI isn’t confined to a single business function. Its value comes from understanding relationships across functions. The better it understands the organisation, the more information it consumes. The more useful it becomes, the greater the temptation to let it act rather than simply advise.

Somewhere along that journey, the conversation quietly changes.The question is no longer whether the AI is intelligent enough.

The question becomes whether the organisation remains in control.

Interestingly, this is the point where model selection almost disappears from the discussion. Whether we were deploying Mistral or evaluating another model rarely changed the underlying architectural questions. Those questions surfaced with remarkable consistency across governments, financial institutions, regulated enterprises and operators of critical infrastructure. Eventually we stopped treating them as implementation details and started treating them as leadership questions. The first is deceptively simple.

Who decides what the AI can see?

Most organisations instinctively answer with identity and permissions. In practice, the question is considerably broader. AI derives value by combining information that was never originally created to sit together. Individually, an engineering roadmap, a commercial forecast and a procurement plan may all be perfectly appropriate datasets. Together, they can reveal strategic intent. Deciding what an AI can see is ultimately about deciding what relationships it is allowed to understand. Those boundaries deserve to be intentional, not accidental. The second question follows naturally.

Who decides what the AI can say?

Enterprise security has traditionally focused on protecting access to information. AI introduces a different challenge. Sensitive information is not always disclosed because a document was exposed. Sometimes it emerges because a model inferred a conclusion from dozens of individually harmless facts. Governing outputs therefore becomes just as important as governing inputs. An organisation should understand not only what knowledge an AI possesses, but also what conclusions it is permitted to communicate. The third question is where responsibility shifts.

Who decides what the AI can do?

Every successful deployment eventually encounters pressure to move beyond recommendations. If the AI can identify an infrastructure issue, why not let it remediate it? If it can prepare a contract, why not let it issue one? If it understands the deployment process, why not allow it to execute the deployment? These are not unreasonable requests. They are exactly what makes AI commercially valuable. They are also the point at which control becomes a more important design consideration than intelligence. The fourth question is one we hear surprisingly rarely.

Who decides when the AI must stop?

Every critical capability eventually requires a means of revocation. Communications may need to be severed. Credentials withdrawn. Workloads isolated. Policies changed. Organisations typically invest enormous effort into granting access and comparatively little into designing how that access disappears under pressure. We found ourselves spending far more time discussing containment than capability. The final question sits beneath all the others.

Who ultimately remains in command?

Technology vendors will change. Models will improve. Regulations will evolve. Infrastructure will move. Jurisdictions will shift. The organisations that navigate those changes successfully will be the ones that retain control over their own systems as everything around them changes. Sovereignty has always been less about where software runs than about who remains able to make decisions when circumstances change.

Looking back, it’s striking how little these conversations depended on AI itself. They were discussions about governance, authority and institutional control that happened to be triggered by AI. The technology merely exposed assumptions that already existed inside enterprise infrastructure.

That observation ultimately shaped how we think about ACRA.

We didn’t build it because AI required entirely new security primitives. Most organisations already operate excellent technologies such as Kyverno, Istio, Cilium, Vault, Falco and OpenTelemetry. We built it because the most useful workloads inside an organisation increasingly require those technologies to operate as a coherent system rather than as a collection of independent controls. AI simply became the first workload important enough to make that obvious to most enterprises.