Anthropic is calling on major artificial intelligence labs to consider a coordinated and verifiable pause in development, warning that rapid advances in the technology could soon allow AI systems to improve themselves faster than society can manage the risks. The creator of Claude said the ability of AI ability to complete tasks on its own has been doubling roughly every four months and it was headed for “recursive self-improvement”, the point at which the technology can improve without human intervention.
“If systems are capable of fully building their own successors, the ways we secure them, monitor them, and shape their behavior all grow much more important,” the company said in a lengthy blog post on Thursday, adding that a pause would allow society to “deal with its immense implications.”
“We are not there yet, and recursive self-improvement is not inevitable. But it could come sooner than most institutions are prepared for,” Anthropic co-founder Jack Clark and Anthropic Institute lead Marina Favaro wrote in the post.
Fears that advanced AI systems may get out of human control and cause societal harm have risen as the technology becomes increasingly capable. Anthropic’s own Mythos model sent shockwaves through industries including banking and software earlier this year with its ability to find vulnerabilities in existing code.
