An Anthropic engineer has publicly resigned from the high-flying company, warning that AI companies are “racing straight to self-improving superintelligence and gambling with our lives.”
Jacob Coxon, who has spent the past three years working on research into how to train AI models, first at OpenAI and more recently at Anthropic, announced his departure in a lengthy social media post on Monday.
“Neither company is acting responsibly,” he wrote. At OpenAI, he said, staff “have not deeply internalized the civilizational stakes.” At Anthropic, he said staff understand the risks well but are “locked in a race to get there first,” based on the theory that no rival company will act as responsibly as they will, so they have the best chance of figuring out how to build superpowerful AI safely.
These dramatic resignations are not uncommon in the AI industry. Over the past few years, several researchers have publicly resigned from AI labs, warning that they are racing headfirst towards catastrophe. Granted, Anthropic, which has long presented itself as the lab most concerned with AI safety, has largely avoided these rebukes, with most of the public criticism aimed at OpenAI.
In this case, though, two current Anthropic employees also publicly confirmed some of Coxon’s assertions. Evan Hubinger, the company’s alignment science lead, wrote: “Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is greater than 10 percent within the next decade.” He added that Anthropic doesn’t yet have a plan to solve alignment for superintelligence, and isn’t clearly on track to get one.
