AI models are rapidly improving – so fast that they may soon be able to develop themselves without human involvement. That’s why Anthropic is warning the AI industry: It needs to build a “brake pedal,” or companies risk losing control of their creations.

"Recursive self-improvement" is a phenomenon which may come sooner than expected, the company says.

RSI is also defined as an “AI system capable of fully autonomously designing and developing its own successor,” per Anthropic’s blog post. “We are not there yet, and recursive…