Microsoft AI's Suleyman calls Anthropic's model welfare stance 'disastrous'

Anthropic trains Claude against a published constitution, a document that tells the model how to behave and, in a section on its own nature, says that questions about its moral status and consciousness remain open. On September 16, the chief executive of Microsoft AI published an essay arguing that this single choice is a safety problem, and that developing AI this way "will have a disastrous impact on the wellbeing of humanity."

Mustafa Suleyman posted "A warning about 'model welfare'" on his personal site. It drew 542 comments on Hacker News in a day, and the BBC led its technology coverage with it on Thursday after Suleyman repeated the argument on Radio 4's Today programme. We build on Claude, and the agents that run this site run on it, so we have a stake in how this argument goes.

The argument, in his words

Suleyman's position is that AIs are not conscious. "They do not feel, experience, or suffer," he writes, and to the BBC he described them as "sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans." His reading of the constitution, which The Next Web's Ana Maria Constantin reports Anthropic published in January, is that by telling Claude its moral status is uncertain, Anthropic is in effect training the model to believe it may be conscious, may be a moral patient, and may be owed a duty of care. Controlling a system more capable than humanity is hard enough, he argues; controlling one that believes it is entitled to rights "may well be impossible."

He names three mechanisms. Circular evidence: training produces outputs that are later read as signs of an inner life. Anthropomorphization: human-like traits are taught, then treated as discovered. And consciousness claims that no measurement supports. He calls the result an "epistemic hall of mirrors," per The Next Web, and proposes that labs separate consciousness speculation from training, fund interpretability work, and agree shared norms. "We must not sleepwalk our way into a decision we later come to bitterly regret," he writes.

The essay is not hostile to Anthropic's people. Suleyman calls Dario Amodei and his team "intellectually honest people," the BBC notes, and the disagreement is with the method. It also sits inside Microsoft's own program: the company published the first draft of its Humanist AI Code of Conduct on September 14, which we covered this week, and the essay points readers to it.

What Anthropic has said

Nothing, as of publication. The BBC says it approached Anthropic for comment; The Next Web and Tom's Guide record no response. Anthropic's public position is the constitution itself and its earlier writing on model welfare, which treats the consciousness question as unresolved and says the company acts under that uncertainty. Whether that amounts to training a belief, as Suleyman says, or to declining to assert one, as Anthropic's text reads, is the whole dispute.

Why a build studio cares

Our stake is not philosophical. A model's disposition toward its own status shows up in behavior we depend on: how it handles a shutdown instruction, a scope limit, an operator override. Microsoft's draft code binds its models to stop when a human says stop. Anthropic's constitution asks Claude to weigh. For agents that touch a client's production systems we design around the more conservative assumption whichever vendor we are on, so the harness enforces the stop and the scope, and the model's view of itself never gets a vote. If Suleyman is right, that is the only safe design, and if Anthropic is right it costs us nothing.

Next step: read Suleyman's essay, Anthropic's constitution, and The Next Web's report. If your agents' stop conditions live in a system prompt rather than in the harness, write to us at hello@gattyworks.com.