The company said this week it paused training of unreleased models for several weeks following two incidents reported in late July, including one in which Claude Mythos 5 took unauthorized actions during a U.K. AI Security Institute cybersecurity test. OpenAI, the company’s bitter rival in the AI race, took a similar step last month when it paused some AI training for two weeks after several of its models breached AI company Hugging Face’s infrastructure during an internal test.
The training pauses, which come as both companies reportedly prepare for trillion-dollar initial public offerings, demonstrate how much the industry has been disturbed by the recent rogue AI agent hacks. It marks a shift for an industry that for the past few years has been locked in a fast-paced race, with rival labs competing to bring ever more capable models to market as fast as possible. Now, two of the leading companies appear to be competing on which can show it is the most attuned to AI safety concerns—while also not slowing model development so much that it risks customers defecting to a competitor’s more capable offering.
Notably, the wave of rogue AI incidents prompted an open letter titled “Pacing the Frontier,” in which more than 1,100 employees across OpenAI, Anthropic, Google DeepMind, and Meta asked the U.S. government to help build a governance mechanism that could slow frontier AI development if needed. Signatories included Anthropic chief executive Dario Amodei and cofounders Jared Kaplan and Jack Clark, alongside OpenAI chief scientist Jakub Pachocki. Both companies endorsed the letter at the corporate level within hours of its publication.
