Anthropic is urging a unilateral slowdown on the development of new AI models until rigorous safeguards can be put in place—with a few caveats. In a blog post published Thursday, the company cited its own internal data as evidence that modern AI systems are nearing the point of “recursive self-improvement”—i.e., being able to refine their capabilities without a human in the loop. “AI that can build itself would be a major development in the history of technology—one that could bring enormous good for the world in science, healthcare, and beyond,” the post, which was written by company cofounder Jack Clark and Anthropic Institute lead Marina Favaro, reads. “But full recursive self-improvement also might increase the risks of humans losing control over AI systems.” The post goes on to suggest that the best way to steer clear of such risks could be a worldwide pause akin to Cold War-era disarmament and non-proliferation pacts, in which countries like the U.S. and Russia agreed to limit their nuclear weapons programs to avoid what came to be known as Mutually Assured Destruction (MAD). “If it were possible to effectively slow the development of this technology to give ourselves more time to deal with its immense implications, we think that would likely be a good thing,” the authors wrote. If this all sounds familiar, you’re not hallucinating. Back in early 2023, nonprofit organization the Future of Life Institute published an open letter calling on all frontier AI labs to enact a six-month pause on the development of powerful new models. It was signed by Elon Musk, pioneering AI researcher Yoshua Bengio, Apple cofounder Steve Wozniak, and tens of thousands of others. While that letter and its luminous roster of signatories did much to kickstart the public conversation around the risks of an AI industry driven by blind capitalist logic, no firm policies were ever enacted to actually make that six-month pause a reality. Just a few months after it was published, Musk publicly launched his own AI startup to compete with OpenAI, Anthropic, Google, and the other big labs.