When the people building the most powerful AI systems on the planet start publicly worrying that those systems might end civilization, it’s probably worth paying attention.
A group of researchers at Anthropic, the company behind the Claude family of AI models, have gone public with stark warnings about the existential risks posed by advanced artificial intelligence. The most striking claim comes from Evan Hubinger, who leads Anthropic’s alignment-science team: he estimates there’s a greater than 10% chance that AI could cause human extinction within the next decade.
The warnings from inside the building
Jacob Coxon, who resigned from Anthropic on September 8-9, took to X to voice his concerns about what he described as a race toward “self-improving superintelligence.” His core argument: neither Anthropic nor OpenAI have adequate safety measures in place to prevent the emergence of uncontrollable “superhuman systems.”
Hubinger’s post went further, putting a number on the fear. His personal probability estimate of over 10% for AI-caused human extinction centers on a specific technical concern: recursive self-improvement. That’s when an AI system becomes capable of enhancing its own capabilities without human oversight, creating a feedback loop that could rapidly produce intelligence far beyond human comprehension or control.
