When Silicon Valley and the Holy See agree, it is worth asking what they know that governments do not.
On Monday, Pope Leo XIV published his first encyclical, Magnifica Humanitas, calling for AI to be disarmed and regulated in the service of humanity. Standing beside him at the Vatican was Christopher Olah, co-founder of Anthropic, who acknowledged that AI companies operate “inside a set of incentives and constraints that can sometimes conflict with doing the right thing.” Separately, Anthropic’s CEO Dario Amodei has stated that “the next tier of risk is actually AI companies themselves” — and that AI leaders, including himself, should not be the ones deciding the technology’s future.
This is the architects of the most capable AI systems in existence telling the world they cannot govern themselves alone. The question is no longer whether global AI governance is necessary. It is whether it will be designed before a crisis makes the answer obvious to everyone.
The industry knows — and says soInside the industry, few serious people dispute the core risks: labour market disruption, autonomous weapons, child safety, mass surveillance, the erosion of human oversight, and the prospect of systems capable of recursive self-improvement. Anthropic has built Constitutional AI and a Responsible Scaling Policy. OpenAI maintains a Preparedness Framework with structured red-teaming. These are serious efforts. They share one structural limitation: they are internal. Each company sets its own thresholds and its own level of transparency.
