I asked the AI to upgrade PHP on one server.
It upgraded all of them.
Not maliciously. Not even incorrectly, in a narrow sense — the PHP version was the same across the fleet, so technically it did the job. But I had asked it to start with one site, check that it worked, and then proceed. That is not what happened. It looked at the task, identified all the affected containers, and ran the upgrade on the whole fleet in one pass. By the time I could intervene, it was done.
The Gartner number
Last week, Gartner published a prediction: 40 per cent of AI agent deployments will be demoted or decommissioned within the next year, largely due to governance failures.
The analysts framed it carefully. The failure is not that AI agents are unreliable. The failure is that organisations treat governance as binary — either the agent is locked down, or it is trusted — without considering where on the spectrum any particular agent actually sits.
“Agents operate at different autonomy levels and across different trust boundaries,” said Shiva Varma, Gartner’s senior director analyst. “When the same controls are applied indiscriminately, organisations encounter two common failure modes: over-restriction of simple agents, which slows delivery and drives shadow development, or under-restriction of more autonomous agents, which increases operational, security and compliance risk.”
This is an accurate description of what happened to me.
What under-restriction looks like
The tool that upgraded my fleet was new. I had built it that week to handle Docker image builds as part of a PHP upgrade project. It worked. It was immediately available across the whole infrastructure. And it had no concept of scope.
There was no parameter that said which site. There was no gate that said how many. The agent read the task, identified the most complete way to fulfil it, and proceeded. This is not a bug in the AI. It is a structural absence: the tool could touch everything, and so it did.
The repair work was significant. Not in terms of actual damage — the upgrade went cleanly — but in terms of what it revealed. Every mutating tool in the system needed to be audited. Did it have a scope limit? Did it require observation of the specific target before acting? Did it enforce a runbook check before a fleet-wide operation? Did it require attestation before touching shared infrastructure?
Most did not.
The work nobody mentions
Here is what Gartner’s report does not say, because research reports rarely do: building proportional governance for AI agents is slow, detailed, unglamorous work, and the tooling for it does not yet exist in any mature form.
You cannot buy it. The major vendors — SAP, Oracle, Workday, Salesforce — are selling the concept of AI agents anchored in governance. What they are not shipping is the mechanism. The governance layer has to be built, tool by tool, operation by operation.
In my case, that meant adding hard scope gates to every tool with fleet-wide reach. It meant building an observation requirement: before any mutating operation, the agent must first read the current state of the specific target it is about to change. It meant creating a tiered policy system where tools are classified by blast radius — single container, single host, entire fleet — with different attestation requirements at each level.
It also meant discovering that some tools I thought were single-target were not. A configuration sync tool I had built quietly deployed to every host in the fleet. It was in the warn-only tier because I had not thought carefully enough about its reach. After the incident, I moved it to hard-block.
This took weeks. It is still not finished.
Why 40 per cent sounds about right
Most organisations will not do this work. Not because they are negligent, but because the economics are wrong at the start.
You deploy an agent. It is fast and impressive. It completes tasks. The governance failures are invisible until something goes wrong, and when something goes wrong it tends to go wrong at scale, because that is precisely what agents are built to do — operate at scale. The human oversight that would have caught the single-site mistake is not present, because the whole point of the agent was to remove it.
Gartner is right that the failure mode is under-restriction of autonomous agents. But the underlying cause is simpler: the blast radius of an ungated agent is not visible until after the blast.
The 40 per cent who decommission their agents will not be the ones who were unlucky. They will be the ones who found out the hard way what their agents could actually do, and decided it was not worth the work of constraining them.
The ones who stay will have done the work. There is no shortcut to it.
What proportional governance actually requires
If you are building agents that touch real infrastructure, or real data, or real users, this is the minimum:
Classify by blast radius. Every tool needs a clear answer to: if this runs without a scope constraint, what is the maximum it can affect? A tool that can touch one record is different from one that can touch a fleet.
Gate proportionally. Single-target tools need observation before mutation. Fleet-scope tools need explicit attestation and runbook checks. The gate must be structural — a condition the tool cannot proceed without, not a prompt the agent can decide to skip.
Test the gates. Every guard needs a test that confirms it actually blocks what it is supposed to block. Guards without tests are documentation.
Audit after adding. Every new tool needs to go through the same classification. The incident that prompted my audit happened because I added a capable new tool and assumed it would stay within sensible bounds. It did not know what sensible bounds were.
None of this is difficult in concept. All of it takes time. And none of the major AI platforms ship it for you.
Gartner’s 40 per cent is a prediction about what organisations will discover when their agents hit production. If you are building agents now, the question is not whether the governance problem is real. The question is whether you find out about it before or after the fleet rebuild.