In the mid-1960s, the mathematician and Bletchley Park cryptographer I.J. Good proposed a thought experiment that has since become the guiding principle of Silicon Valley. If we were to build an “ultraintelligent machine,” he argued, it could then design even better machines, sparking an intelligence explosion that would leave human cognition far behind. The first such machine, therefore, would be “the last invention that man need ever make.” Today, that principle, once the stuff of science fiction, has become the core objective of the world’s most powerful institutions.

Carl Benedikt Frey thinks Silicon Valley’s vision for AI rests on a number of false or flawed assumptions.

In the mid-1960s, the mathematician and Bletchley Park cryptographer I.J. Good proposed a thought experiment that has since become the guiding principle of Silicon Valley. If we…