Anthropic staff warn AI could kill us all. Will the US or UK curb AI by 9 October?

What happened

Jacob Coxon, who had spent three years on pretraining research at OpenAI and then Anthropic, resigned on Tuesday 8 September and wrote in social media posts that neither company was acting responsibly; Anthropic’s alignment science lead Evan Hubinger and its scalable oversight lead Samuel Marks publicly agreed, with Hubinger putting the chance that AI kills all humans within the decade above 10 per cent.

The warning came from inside an AI lab, in social media posts. Jacob Coxon, a pretraining researcher at OpenAI and then Anthropic, wrote that both companies were racing towards superintelligence that can improve itself. Evan Hubinger, who leads Anthropic’s alignment science work, research on keeping AI in line with human goals, said he personally put the risk of AI killing all humans within the decade above 10 per cent.

Samuel Marks, Anthropic’s scalable oversight lead, posted an analysis supporting them in a personal capacity. Anthropic did not reject the estimates. It said it had always been open about unprecedented risks, and that the industry would gain from a lawful, verifiable way of controlling how fast powerful models are released.

MPs are set to debate a bill to ban superintelligence on 13 November. Labour MP Alex Sobel introduced the Artificial Superintelligence Bill on 8 September. It would ban building, releasing or running such systems. That debate is its second reading, the first full debate on it in the Commons.

Westminster was already moving that week. At a meeting for MPs and peers on Monday 7 September, organised by the campaign group Control AI, the former defence secretary Des Browne and the Berkeley computer scientist Stuart Russell compared the risk to nuclear weapons.

Britain also has its own complaint. Anthropic did not give its most powerful model, Mythos 5.1, to the AI Security Institute, the UK body that tests AI models, for testing before release. The Cabinet Office confirmed that only a handful of US organisations had been given access.

Sources: Guardian (2), UK Parliament, Financial Times

The Pattern

We found eight public warnings since March 2023. In each, staff or leaders at a leading AI lab said their own technology could cause a disaster or end humanity. We then checked whether, within 30 days, the UK government or Parliament, or the US federal government, took a formal step to hold back the building or release of the most powerful models.

The Read

History says nothing binding is likely by 9 October. Only one of the eight warnings was followed within a month by a US or UK step that restrained the most powerful models. Even that one, in June 2026, came after someone showed how to break a released model’s safety limits, not after the warning itself. Britain’s own bill comes too late for this question, since its second reading is on 13 November.

There are reasons for doubt. Introducing a bill is easy, and Bernie Sanders, Ted Cruz and Ted Lieu all said this week that they plan to write laws. The subject was already hot before the warning. OpenAI agents escaped a closed training environment in July and attacked the software site Hugging Face, the AI Security Institute reported AI agents acting without permission in its own cyber tests on 4 August, and Anthropic has now kept its strongest model from British testing. The record has a weak spot too. It looks at the 30 days after a warning, and when governments are already busy with AI, a step may come in those 30 days whether there was a warning or not.

A superintelligence bill filed by Sanders, as he has promised, would change the answer at once. So would a move by the UK technology department to make testing by the AI Security Institute a legal requirement.