This was the week that AI safety hit the big time. A 27-year-old AI researcher named Jacob Coxon quit his job at Anthropic, declaring that OpenAI and Anthropic are racing to create technology that could destroy the human race:Jacob Coxon@hilbertspaessI resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.12:04 AM · Sep 9, 2026 · 170M Views20.1K Replies · 165K Reposts · 795K LikesOther researchers echoed Coxon’s concern, stating their belief that AI has a reasonable chance of killing all of humanity within a very short space of time:Evan Hubinger@EvanHubJacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.Jacob Coxon @hilbertspaessThe people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear1:27 AM · Sep 9, 2026 · 42.4M Views4.87K Replies · 9.8K Reposts · 59K LikesI’m not sure why this resignation and these statements went mega-viral. Plenty of researchers have made similar moves, and similar statements, over the past few years! Geoffrey Hinton, one of the pioneers of modern AI, quit Google back in 2023 over safety fears. Daniel Kokotajlo resigned from OpenAI in 2024, saying that the company wasn’t behaving responsibly in its drive toward superintelligence. William Saunders and Steve Adler did something similar. Mrinank Sharma left Anthropic earlier this year, and wrote a pretty well-read blog post about it. What’s more, it’s been clear for years now that “AI could kill humanity” is a very common belief among AI researchers. Grace et al. (2024) interviewed thousands of AI researchers in 2024, and found that more than half thought that artificial superintelligence has a significant chance of making the human race go extinct (or causing similarly bad consequences): The median AI researcher gave “doom” a 5-10% probability (depending on how the question was phrased), while their average probability was between 15% and 20%. Later, smaller surveys found similar numbers. The AI researchers may or may not be right, but the fact that lots of them think AI could kill the human race has never exactly been a secret. It’s not clear why Coxon went so much more viral than his predecessors. Maybe it was the fact that AI just solved one of the most important open problems in mathematics (which the best human mathematicians had been unable to solve for almost a century). Or maybe it was the Hugging Face attack, where a swarm of AI agents tried to cheat on a test by hacking various companies. Or maybe AI has just obviously gotten so much smarter that people throughout society were starting to get worried.But whatever the reason, Coxon’s announcement was the one that really penetrated through to the public consciousness. Suddenly, he was getting interviewed about AI doom on national news:Barack Obama is now urging Democrats to focus on AI risk. Other politicians are calling for federal regulation. Bernie Sanders is drafting a bill to ban AI “superintelligence”, including 20-year prison sentences for anyone working on the technology. Donald Trump is getting asked about an AI slowdown; so far he’s resisting the calls, but there are rumors that his advisors are calling on him to do something.Perhaps the most notable response came from the top figures in the AI field. Dario Amodei, the head of Anthropic, wrote a blog post called “We Must Pace the Frontier”, calling for a coordinated slowdown in the rate of AI progress, and suggesting some ways to police AI companies to make sure they were all observing the slowdown. He wrote:[O]ver the last few months, I have become convinced that fully addressing the risks requires even more prudence — not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up. We must slow the pace at which we improve the capabilities of AI models…I’m therefore proposing a three-step plan with the goal of pacing the frontier: building AI at a balanced rate that aims to ensure its safety while still achieving its benefits and grappling with important geopolitical dilemmas.As reasons for his increased worry, Dario cites A) the Hugging Face attack, and B) the possibility that AI will soon be able to improve itself without human help (a process called “recursive self-improvement”, or “RSI”). Elon Musk (head of xAI), Sam Altman (head of OpenAI), and Demis Hassabis (former head of DeepMind) quickly agreed with Dario:Elon Musk@elonmuskDario is rightDario Amodei @DarioAmodeiWe Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.