Jacob Coxon, who spent three years working on pretraining research for large AI models at OpenAI and Anthropic, has quit Anthropic. His accusation is that both companies are gambling with the survival of the human race.
Anthropic employee Evan Hubinger puts the odds at more than ten percent that a misaligned superintelligent AI could destroy humanity within the next decade. His statement came in response to the departure of Jacob Coxon, who led pretraining work at Anthropic and previously at OpenAI.
Anthropic AI safety researcher Evan Hubinger says there's a greater than ten percent chance AI destroys humanity in the next ten years. | Image: via X
Coxon believes current AI systems are on the verge of becoming superhuman. "These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources," he writes, adding that the progress is obvious and it isn't slowing down.
Why keep building when the danger is known?
