The narrative around AI suddenly took an existential turn over the past week as leading voices sounded the alarm over humanity’s future, and the fear of an imminent calamity has sharpened those warnings.
On Saturday, Anthropic CEO Dario Amodei called on the industry to slow down development, saying that AI has been advancing “drastically faster” since the summer, driven primarily by AI’s ability to upgrade itself.
If left unchecked, this so-called recursive self-improvement could outrun the ability of humans to control AI, he explained in a blog post.
Amodei also pointed to the hack of Hugging Face by hundreds of autonomous AI agents, warning that a similar swarm armed with greater capabilities could have caused “catastrophic damage.”
“Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails,” he wrote.
