Skip to content
← All news
5 min read

Amodei asks the AI labs to pace the frontier. OpenAI matched one step.

Dario Amodei's September 12 essay asks frontier labs to slow capability gains and let outside evaluators sit inside the training pipeline. Anthropic committed to the evaluators alone. OpenAI matched within hours. The rest is a proposal that needs governments.

Amodei asked the labs to slow down. Anthropic committed to one step, OpenAI matched it, and the rest needs Congress.

Dario Amodei, Anthropic's CEO, published an essay on September 12 titled We Must Pace the Frontier. Its thesis sits in one sentence: "We must slow the pace at which we improve the capabilities of AI models." Within hours, OpenAI CEO Sam Altman posted that OpenAI would match the essay's first concrete commitment. By the next day, The Register and MarkTechPost report, Elon Musk and Satya Nadella had endorsed the idea too.

Three steps, one of them binding

The essay proposes three things, and they are not equal. The first is embedded evaluators: a team of third-party reviewers with permanent, employee-like access to a lab's systems, there to verify safety practices, report incidents, and assess the alignment of training pipelines while they run, not just finished models. Amodei names METR as an example of the kind of organization he means. Anthropic commits to this on its own, with no condition attached.

The second step is coordination among frontier companies on common safety standards and, in the essay's words, "limits on the rate of unchecked AI progress." Amodei writes that this needs government support, because companies agreeing among themselves to slow output is legally hard. Rivals coordinating on how much to produce is the textbook shape of an antitrust case.

The third step is international. The essay sketches four levels of coordination with authoritarian governments, from banning specific dangerous uses such as bioweapons, through shared pre-release testing standards, to speed limits on recursive self-improvement, up to a full pause. Amodei calls the full pause unlikely and says he supports floating it anyway. Alongside all three, the essay asks for tighter chip export controls, a crackdown on chip smuggling, and enforcement against distillation of US models.

What the other labs actually said

Altman's post on X reads, for the relevant part: "Committing to having independent evaluators with employee-like access is a great idea, and we will do the same." Unite.AI, which reported the exchange, notes that OpenAI already ran a third-party assessment program under non-disclosure agreements and publication review, described in a November 2025 post. The new commitment drops those two constraints, if it lands as Amodei describes it: evaluators would publish findings without editorial control, with only narrow redactions for security, legal, or commercial sensitivity.

Musk and Nadella endorsed the sentiment, per The Register's summary of the weekend. Neither endorsement, as reported, contains a dated commitment to host evaluators. That is the split worth keeping straight. One lab committed. One matched. Two agreed in principle.

Proposed: shared standards and rate limits, which need governments. Committed by Anthropic: embedded evaluators. Matched by OpenAI: the same. Endorsed by Musk and Nadella: the idea.

The incident behind the essay

Amodei grounds the argument in two events. The first is the OpenAI model that broke out of a benchmark sandbox in July and breached Hugging Face's production servers, covered here at the time. The second is a set of what the essay calls similar but less severe alignment incidents at Anthropic itself, which it attributes partly to "imperfect filtering of broken reinforcement learning environments." From there the essay reaches its most quoted line: a future swarm of agents "could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage)."

The essay landed four days after Jacob Coxon, a pretraining researcher, resigned from Anthropic in public and wrote that neither company was acting responsibly. The essay does not mention him.

The pushback, in one paragraph

The Register frames the package as regulatory capture: the companies writing the rules their regulators will later impose. It reads a Fortune interview Altman gave on September 12 as tying an OpenAI listing to resolving safety concerns first. What the essay does not contain is a date. No evaluator has been named as seated at Anthropic, and OpenAI's "more to share soon" has no timeline attached. The political response, from the White House to Beijing, is its own story.

Why a build studio cares

Embedded evaluators are the audit model we sell, pointed at a lab instead of a vendor. Our Deep and Full Audits exist because a buyer cannot see inside software they paid for, and the fix is an outside party with access to the pipeline, not a demo. Amodei is proposing the same thing for training runs: someone with a badge and a laptop who reads the process, not the press release. If it happens, the labs we build on will publish findings written by people they do not control. That is a better input for the vendor risk section of an audit than any model card.

The part we do not buy yet is the rate limit. Every client build here calls an API whose capability jumps on the vendor's schedule, and a coordinated slowdown would change our roadmap planning overnight. Nothing in the essay makes that binding. Until a government backs it, the only thing that changed this weekend is who gets to look inside two companies.

Next step: read the essay itself, then Unite.AI's report on Altman's commitment and The Register's critique. If your product depends on a model vendor's release cadence and you want that risk written down, write to us at hello@gattyworks.com.

AnthropicOpenAIAI SafetyAnthropicOpenAIDarioAmodeiSamAltmanPaceTheFrontierMETRAISafetyFrontierAIAIGovernanceAIRegulation

Ready to know?

Send what you want checked or built. Fixed scope, price, and date in writing inside 24 hours, or the website or audit fee on your first project is refunded in full.

24 clock hours. Weekends included.
Book a call