The AI safety community is experiencing something of an identity crisis. Researchers who spent years sounding alarms about the existential dangers of advanced AI systems are now, in some cases, arguing that those same systems should be made more widely available.
The safety hawks are evolving
To understand why this matters, you need to rewind a few years. In March 2023, a group of heavyweight researchers, including Yoshua Bengio, co-signed the Future of Life Institute’s open letter calling for a six-month pause on training AI models more powerful than GPT-4. The letter wasn’t subtle. It framed unchecked AI development as a civilizational risk.
Geoffrey Hinton, often called the “godfather of deep learning,” went even further. He compared the open-sourcing of powerful AI models to the proliferation of nuclear weapons.
The Claude Mythos precedent
