SynopsisAI that can build itself would be a major development in the history of technology, but "full recursive self-improvement also might increase the risks ‌of humans ⁠losing control ⁠over AI systems," the AI startup said.Anthropic said on Thursday frontier AI developers should establish a coordinated, verifiable way to slow down or temporarily pause development if advanced systems begin improving themselves faster than society can manage the risks.AI that can build itself would be a major development in the history of technology, but "full recursive self-improvement also might increase the risks ‌of humans ⁠losing control ⁠over AI systems," the AI startup said."If systems are capable of fully building their own ​successors, the ways we secure them, monitor them, and shape their behavior all grow much ​more important."As an example, Anthropic said that as of May, more than 80% of the code merged into its codebase was authored by Claude.It would ​be "good for the world to have the option ⁠to slow ‌or temporarily pause frontier AI development to enable societal ​structures and ​alignment research to keep up with the advance of the ⁠technology," the company said.However, it cautioned that unilateral or ​poorly coordinated slowdowns could backfire if less cautious actors ​continue advancing, potentially reducing overall safety.It highlighted that a meaningful pause would require agreement among "multiple well-resourced labs" operating at the technological frontier, as well as rules on what conditions would trigger or lift such a pause and who would oversee it.A unilateral pause by a single company would be ‌easier to implement, Anthropic added, but would have limited impact, primarily shifting leadership rather than fostering broader global deliberation.Its research arm, ​Anthropic Institute, ​plans to study and ⁠help build systems that would be necessary to support a slowdown.In the coming months, Anthropic plans to convene discussions involving policymakers, researchers, civil society groups and ​other AI firms to examine key questions.These questions include how to manage AI-related risks such as recursive self-improvement and how to improve mechanisms for coordination.Last month, Anthropic concluded a fundraising round that valued the company at $965 billion and confidentially filed for a U.S. initial public offering on Monday. ...moreElevate your knowledge and leadership skills at a cost cheaper than your daily tea.Subscribe Now