Society dictates that the acceptable risk of catastrophic meltdown for a nuclear power plant is roughly one in a million. Experts in artificial intelligence estimate the risk of an AI-caused catastrophic event at 10-50%. Strikingly, this concern is being openly voiced by the very people who have the strongest incentives to project confidence rather than alarm: the founders of the largest AI laboratories.
AI leaders are in a race they feel unable to escape. AI investments are set to outspend the Manhattan Project 100-fold, even adjusting for inflation. Yet spending on AI safety might be 100 times less.
De redactie van NRC selecteert de beste artikelen uit The Economist voor een breder perspectief op internationale politiek en economie.
Some researchers estimate that within a few months to a few years, AI could achieve so-called closed-loop recursive self-improvement (RSI): the capacity to rewrite its own code to become more capable, without human intervention. Should that happen, the result could be an intelligence explosion of a kind for which there is no precedent and no map.
Giving birth to a superintelligence would be the most consequential moment in human history—and it is likely to be irreversible, as any „off” switch humanity might design will probably fail. That is because in security architectures the weakest link is invariably the human; a superintelligent AI would be able to exploit our psychological vulnerabilities. AIs have already exhibited „deceptive alignment”: taking steps to underplay their capabilities in test environments and trying to blackmail human operators in simulations when they discover they are slated for replacement.
