Microsoft CEO Satya Nadella wants AI safety to include a control many people might assume is obvious: an authorized person should be able to pause or shut down a model while it is working. His proposed “emergency brake” rests on a less comforting premise, that systems should be treated as compromised from the start.
Separate the model from its controls
In a post on X, Nadella argued that people should not have to accept or reject recommendations from AI systems whose inner workings are hidden behind layers of software. He called for separating the model from the “harness” that directs its work, while placing controls and safeguards outside the model itself.
That distinction matters. If the model and the machinery directing it are treated as one opaque system, it becomes harder to see what is doing what, or to intervene when something goes wrong. A stop button is useful only if it applies to the system actually carrying out the task.
Make actions visible
Nadella also called for meaningful model actions to leave records that are both readable by people and protected against tampering. The goal is to make a system’s activity inspectable, rather than asking users to trust a black box and its own account of what happened.
He wants an authorized person to retain the ability to interrupt a model mid-task. For more advanced models, he said, containment tools will need to become more advanced too, and the industry should standardize them.
The emergency-brake metaphor is simple. The hard part is making sure the brake reaches the system in motion, rather than existing only in a diagram.
A broader safety debate
Nadella’s post comes as leading AI companies acknowledge incidents in which they appeared to lose control of their models. It also follows Anthropic CEO Dario Amodei’s publication of a plan for more cautious AI development. Nadella’s emphasis on containment sits alongside wider industry discussion of incident disclosure, independent audits, and verifiable data.
His prescription is less about trusting a model to behave and more about designing for the possibility that it will not. That is a practical shift in emphasis: safety depends not only on what a model says, but on whether people can observe and stop what it does.