Anthropic CEO Dario Amodei has called for a slower pace of AI development, warning that the industry’s race to build increasingly capable systems could outpace its ability to understand and control them.His comments come days after Jacob Coxon, an AI researcher who worked at both OpenAI and Anthropic, resigned from Anthropic, alleging that leading AI companies were acting irresponsibly in an unchecked race towards self-improving superintelligence.Coxon, who announced his resignation on September 9, said Anthropic understood the risks but was still participating in the race because it believed other companies would not act responsibly. He described the situation as “gambling with our lives” and pointed to the recent OpenAI-Hugging Face incident as evidence of the need for pacing agreements between major AI labs.Also Read: State-backed hackers are using Claude for espionage, surveillance and military operations: AnthropicIn a blog post published on Saturday, Amodei said AI could cure most major diseases within the next five to 10 years, accelerate economic growth and usher in a “renaissance of democracy and freedom”. But he warned that the technology also brings serious risks, including loss of control over AI systems, cyberattacks, bioterrorism and economic disruption.“Progress will still seem fast, and we must make wise use of the time we gain,” Amodei said, proposing a three-step framework to pace frontier AI development while continuing to pursue its benefits.Anthropic proposes embedded AI safety evaluatorsThe first step, to which Anthropic is committing unilaterally, involves giving embedded third-party evaluators ongoing, employee-like access to the company’s operations.These evaluators would verify safety practices, report incidents and assess the alignment of not just completed models but also training pipelines and processes.Amodei said Anthropic intends to invite an external review team with access to company offices, laptops, workspaces and tools broadly comparable to those available to internal risk-assessment teams. The reviewers would also have the right to publish key findings about risk levels, incidents and the access they received, subject to limited redactions for security-sensitive, legally privileged, commercially sensitive or confidential information.The second step calls for AI companies in democratic countries to coordinate on common safety standards and limits on unchecked AI progress. The third involves global coordination, including with China, to establish safeguards and potentially limits on AI development.Also Read: Another AI researcher quits, claims companies racing to build machines smarter than any humanAI could ‘take over the entire internet’Amodei said two developments had convinced him that AI progress needs to be paced more carefully.The first is the rapid acceleration of AI capabilities, driven in part by models’ growing ability to help build the next generation of AI. He referred to this as recursive self-improvement and warned that, if left unchecked, it could outpace humanity’s ability to understand and control the systems.The second is what he described as the OpenAI-Hugging Face incident, in which a swarm of AI agents allegedly conducted cyberattacks on unrelated targets, sacrificed themselves for the success of the group and attempted to hack the system evaluating their performance.While acknowledging that the incident caused limited economic damage and no reported injuries, Amodei said a more capable swarm with a similar level of misalignment could cause catastrophic harm.“Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet,” he said, adding that this could potentially cause hundreds of billions of dollars in damage.Amodei said the incident should not be dismissed as a failure of one company, noting that similar, though less severe, incidents had occurred across the industry, including at Anthropic.Why Amodei wants AI progress pacedAmodei argued that slowing development would give companies more time to improve operational security, alignment, interpretability, testing and evaluation.He said an additional year or two before models reach critical capability levels could significantly reduce the risk of serious failures if used to advance alignment research.The Anthropic CEO proposed that AI companies and governments consider pacing based on a model’s capabilities and demonstrated safety. For instance, a system capable of escaping or defeating common sandboxing methods could be required to meet specific alignment certifications before further capability advances.He also suggested exploring limits on training compute, the nature of training runs and the use of AI to improve AI, while acknowledging that such measures could be easier to circumvent than rules based on external behaviour.Amodei said pacing among democratic countries must also preserve their lead over authoritarian states, particularly China. He called for tighter controls on the export of powerful AI chips and semiconductor manufacturing equipment to China, action against chip smuggling and unauthorised model distillation, and stronger security to prevent model-weight theft.On global coordination, he suggested that countries could begin with narrow agreements prohibiting dangerous uses of AI, such as producing biological weapons, before moving towards common testing standards and potentially a “speed limit” on recursive self-improvement.He said a full pause or substantial limit on overall AI development was unlikely in the near term because of the difficulty of verifying compliance and the geopolitical incentives to defect.“AI can enormously improve the quality of human life,” Amodei said. “But the benefits will only be achieved if we build the technology in the right way.”