Will you be flagged at the border? Will your mortgage application be approved? During wartime, whose neighborhood would a weapon system target? These are moral choices—about harm and fairness—and they used to be made by people.
Now moral choices like these are made by artificial intelligence (AI) and by the companies developing it. Not the government, not the public, but corporations.
Chris Olah, co-founder of the AI company Anthropic and a self-described atheist, recently sat beside Pope Leo XIV at the Vatican and said his own industry cannot be trusted to govern itself. "Some might believe that matters of AI are best handled by computer scientists like myself," he said. "They are mistaken."
Olah was echoing the pope's first encyclical, "Magnifica Humanitas: On Safeguarding the Human Person in the Time of Artificial Intelligence," which warns that AI must serve humanity rather than concentrate power.
It is clear that AI needs an independent regulator with the power to say no in the same way that authorities can refuse a new drug or block a nuclear reactor. Ordinary people, not just technical experts, need to set the moral standards this regulator enforces.
