Who can switch it off? Eight controls to have before AI agents act alone
Up to a certain point, a person approves everything an AI system does. Past that point the system acts first. Here is what needs to be true before you cross the line.
The short version
- The moment an agent can act without someone approving each step, eight controls stop being nice to have.
- They are about ownership, limits, visibility and the ability to stop and undo.
- Being strong elsewhere does not make up for one that is missing.
The line that matters
Sociotek describes six levels of autonomy. At the lower three, a person decides or approves every action. From the fourth, which we call A3, the agent completes work on its own and only hands the unusual cases to a person.
That is a sensible way to work, and it changes what you need in place. If nobody is approving each step, the protection has to come from how the agent is set up and watched.
The eight controls
- A named owner. One person or team answers for each agent, from the day it starts to the day it is retired.
- Its own identity. The agent is known to your systems as itself, so its actions are not hidden behind someone's login.
- Limited access. It can touch only what its task needs.
- A trail. You can reconstruct what it did and why.
- A second check. For actions that matter, something other than the agent verifies the result.
- An off switch. The right people can pause or stop it quickly, and someone has tried.
- A way back. Mistakes that matter can be corrected or undone.
- Recourse. A person affected by its decision can question it and be heard.
The full Sociotek diagnostic adds two more: rules for the data an agent uses and what it remembers, and a rehearsed plan for when something goes wrong.
Why an average will mislead you
It is tempting to score all of this and take comfort from a good overall number. Resist that. An organisation with excellent training and governance and no way to stop a misbehaving agent is not mostly ready. These controls work like the brakes on a car: the quality of the engine does not make up for them.
What to do
- List the agents doing real work in one part of your business. If that takes more than an afternoon, you have found the first problem.
- Next to each, write the owner's name. Leave blanks blank.
- Ask who can switch each one off, then ask when that was last tested.
- For any agent that acts without approval, go down the eight controls and mark each yes, no or do not know. Treat do not know as no until shown otherwise.
Words explained
- Autonomy
- How much an AI system may do without a person stepping in.
- Bounded operator (A3)
- An agent that completes a defined piece of work on its own and passes exceptions to a person.
- Control
- Something you put in place so that a risk is prevented, noticed or corrected.
- Recourse
- A way for someone affected by a decision to challenge it and have it looked at again.
All read-outs and the plain-language glossary