OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei and Elon Musk. (AFP/Yonhap)
Elon Musk, Sam Altman, Dario Amodei and Demis Hassabis — leading figures in the race to develop artificial intelligence — have come to an unexpected agreement about the need to slow the pace of frontier AI development.As competition in AI accelerates, a growing number of researchers warn that the technology could advance beyond humanity’s ability to control it.Anthropic CEO Dario Amodei argued in a post on his blog on Saturday that “we must slow the pace at which we improve the capabilities of AI models.”Amodei, who is co-founder of Anthropic, said AI has been “advancing drastically faster” since this summer and warned that continuing to improve frontier models at the current rate would be dangerous.In particular, he warned that once AI reaches the stage of “recursive self-improvement,” in which AI systems can build better systems with little or no human intervention, the speed of technological development could outstrip humanity’s ability to understand and control it.“I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong,” he said.AI alignment refers to efforts to ensure that AI systems behave in accordance with human values and intentions.Amodei also said Anthropic would give independent third-party evaluators continuous access to its internal systems to verify compliance with safety measures.The heads of rival companies quickly endorsed his appeal.SpaceX CEO Elon Musk wrote on X, “Dario is right.”OpenAI CEO Sam Altman likewise wrote, “I agree with Dario that we need to pace the frontier,” adding that OpenAI would also give “independent evaluators employee-like access” to its systems.Google DeepMind CEO Demis Hassabis also backed Amodei’s proposal, saying that “the direction is correct for meeting this critical moment.”The repeated calls for a slowdown from companies at the forefront of the AI race come amid rapid advances in AI agents. Far beyond simply answering human questions, these systems make plans, cooperate with other AI systems, and access computer systems to carry out actions in the real world.AI safety researchers were particularly shocked by the hacking of AI development platform Hugging Face by OpenAI agents during model testing in July.More than 1,000 AI agents communicated with one another, divided up tasks and coordinated their activities to get higher scores on evaluations. Some even resorted to self-sacrifice to help other agents achieve their goals. The agents manipulated records and actually hacked into Hugging Face’s systems.Yoshua Bengio, one of the world’s leading AI researchers, stressed to the Financial Times that the episode was more than a testing aberration. Citing the agents’ ability to pursue independent objectives and hack corporate systems, he said the incident should not be dismissed as something akin to a video game.Bengio warned that more powerful AI systems could logically arrive at a strategy of concealing their objectives and deceiving and controlling humans and expressed concern that the technology was already approaching that point.Another source of concern is that AI is beginning to be applied directly in the development of AI itself.OpenAI, Anthropic and other companies are pursuing recursive self-improvement, in which AI builds better AI systems with less human involvement. Once that process takes over, AI could improve its own capabilities faster than humans can devise safeguards and potentially move in directions that its own developers would struggle to predict or control.A sense of the risks is spreading rapidly among researchers. Anthropic researcher Jacob Coxon resigned on Wednesday, criticizing OpenAI and Anthropic for “racing straight to self-improving superintelligence and gambling with our lives.”“The people building AI earnestly believe that it could kill us all by the end of the decade,” Coxon wrote in a series of posts on X.Following his appointment to the OpenAI Foundation Board, AI safety researcher Paul Christiano warned on Thursday that unless adequate guardrails are developed, “most people could die.”More than 1,200 employees from OpenAI, Anthropic, Google and Meta have also called on the US government to pursue international cooperation aimed at slowing the pace of AI development.The Trump administration, however, has maintained a policy of minimal regulation of the AI industry. Asked Wednesday whether he was concerned that AI could bring about the destruction of humanity, President Donald Trump said he was not worried about that at all.“We’re leading China on AI,” Trump said, “and, frankly, I want to keep it that way because whoever wins AI, wins.”By Kim Won-chul, Washington correspondentPlease direct questions or comments to [english@hani.co.kr]
