Anthropic calls for global pause in AI development before humans lose control
Anthropic PBC has sounded the alarm on the rapid pace of artificial intelligence development and is calling on the world’s top AI laboratories to consider slamming on the brakes. In a blog post today, the company warned that AI systems are getting close to the point where they may soon be able to improve themselves without human oversight, and said reaching this threshold could lead to massive societal disruption.
The startup’s solution is for a globally coordinated agreement to temporarily pause or at least slow down the pace at which new frontier models are being developed. In the blog post, Anthropic’s head of internal research Marina Favaro and head of policy Jack Clark argued that a pause would provide the world with the breathing room it needs to adjust to the pace of AI’s rapid growth.
The two authors said that model advances appear to be getting closer to a theoretical concept known as “recursive self-improvement,” which refers to AI systems that are capable of independently improving themselves and expanding their own capabilities, simply by writing their own code. They acknowledge that this hasn’t happened yet and isn’t necessarily inevitable, but they also warn it could happen – much sooner than anyone is prepared for. According to Clark, there’s reason to believe that some models could be capable of recursive self-improvement within just two years.
