Anthropic, a company that builds frontier AI models, has written a bill that would make it harder for the government to shut down frontier AI models. It also asks Congress to impose safety rules on companies like Anthropic. The proposal is a confidential draft that Anthropic has shown to American lawmakers, and a news outlet has since obtained a copy. It is a company proposal, not legislation, and nobody in Congress has yet agreed to anything.
It is an unusual pitch. Regulated industries usually ask for fewer rules. Here, a company is asking for a thorough rulebook, with a very high bar for the one power that matters most to its business: the power to pull the plug.
The trigger: a very high bar
Under the draft, the government could seek to block the release or use of a model only if it poses a foreseeable and imminent threat of one of two kinds of harm:
- Financial damage: more than $1 billion.
- Human harm: the death or serious injury of more than 50 people.
The threat would also have to fall into one of three categories: the creation or spread of biological weapons, a model slipping out of its developer's control, or a cyberattack.
That is a short list. A model that merely behaves badly, spreads misinformation or helps with ordinary fraud would not qualify under this language. The draft reserves the shutdown power for the scenarios that dominate AI safety debates.
Who gets to pull the plug
The second half of the mechanism is about who holds the switch. The Department of Commerce could not halt a model on its own. It would need a ruling from a federal court in the District of Columbia.
The draft also bars other federal agencies and officials from suspending the development, deployment or internal use of such models. So the route to a shutdown would run through one department and one court, and nowhere else.
This is the part that will probably draw the most debate. Supporters can call it due process. Skeptics can point out that a company has drafted rules limiting how much the government can interfere with that company. Both readings are available, and the draft does not settle which one is right.
Independent evaluators, with a paycheck from the people they check
The draft creates a role for independent evaluators. The mechanics, as described:
- Registration: an evaluator's assessment must be registered with the Department of Commerce.
- Periodic checks: those responsible would review developers and their models from time to time.
- Payment: evaluators may be paid by the companies they examine, as long as the fee does not depend on the outcome of the review.
- 72-hour clock: if an evaluation finds an imminent threat of the kind the bill covers, the evaluator must notify the agency within 72 hours of finishing the review.
The fee provision is a pragmatic compromise. Someone has to pay for these reviews, and the company being reviewed is the obvious candidate. The safeguard is that the money cannot be tied to a verdict. Whether that is enough to keep evaluators independent in practice is a fair question, and the draft's text as reported does not answer it.
The safety side of the bargain
The bill is not only a shield. It also contains obligations for developers of frontier models:
- mandatory safety rules
- published transparency reports
- protection for whistleblowers
- reporting of critical incidents to the Department of Commerce
An Anthropic spokesperson said the document reflects the company's Advanced AI Framework, a set of safety requirements for powerful AI models that the company has been promoting to federal and state authorities since June. According to the spokesperson, Anthropic considers the measures reasonable and urges Congress to adopt them.
The part that touches the states
The draft would also limit some state requirements. Specifically, it would restrict state rules on disclosing risks to the public, on independent evaluations, and on reporting critical incidents.
So the offer is a single federal standard in exchange for narrower state rules. Companies have long favored this arrangement because one rulebook is cheaper to follow than fifty. Whether states will happily give up their own requirements is another matter, and the report does not say how they might respond.
Where this sits in the wider debate
The draft arrives while other federal ideas on AI regulation are under discussion. Some senators are working on measures aimed at catastrophic AI risks. Meanwhile some politicians and safety advocates want tougher oversight, or even a pause in the development of frontier systems.
Against that backdrop, Anthropic's text sets out narrower grounds for government intervention than approaches that call for broad review of models before release. That is the clearest dividing line in the story. One camp wants regulators to look first and approve later. This draft lets companies ship, imposes reporting and evaluation duties, and keeps intervention for a short list of severe, imminent dangers.
What is still unknown
The available report leaves a number of questions open:
- Legislative path: there is no word on who might sponsor the bill or when it could be introduced.
- Definitions: how an "imminent" threat would be judged is not detailed.
- State reaction: the report does not cover how state officials view the proposed limits.
- Court process: the procedure for obtaining a D.C. court order is not described.
What can be said is limited. A company that builds powerful AI has put in writing what it thinks the rules should be, including the conditions under which it could be stopped. Congress now has to decide whether to take those conditions as a starting point, and how to fit one national standard together with what individual states already require.
It is, at least, a rare case of a company volunteering a definition of the worst case. Drafting the exit conditions yourself is a bold way to start a negotiation, and lawmakers will no doubt notice who held the pen.