Intelligence is a slippery thing. For millennia, humans assumed it flowed through a fluid, or one continuous sheet crumpled inside the skull. Microscopes later revealed a dizzying network of individual neurons, sculpting intelligence from electrochemical noise to give our brains a certain je ne sais quoi.
In the late 1950s, psychologist Frank Rosenblatt designed the perceptron, a brain-inspired algorithm that adjusts the relative strength of its units’ connections with experience. The New York Times described it as “the embryo of an electronic computer” that would one day “be able to walk, talk, see, write, reproduce itself and be conscious of its existence.”
By 1971, the year Rosenblatt drowned in a boating accident, this “connectionist” approach to artificial intelligence had been overpowered by the field’s attachment to hand-coding clever rules based on human knowledge. (In the 1970s, for example, AI researchers asked doctors how they spot bacterial infections, and coded a diagnostic tool by meticulously transcribing hundreds of human-inspired if-then rules. Any expertise doctors couldn’t put into words — one might call it vibes — was inherently left out.)
Fast-forward to today, and every large language model is powered by deep learning, an incredibly connectionist approach. It’s perceptrons all the way down, stacked more than Rosenblatt could imagine. AI researcher Rich Sutton called it the Bitter Lesson: general, brute-force computation usually works better than trying to recreate our own thinking from scratch. And sometime after the 2017 introduction of the transformer, AI developers found another way to piss off old-school purists who believe intelligence has to be intentionally designed: make models better by making them bigger. Hundreds of billions of parameters, the numerical knobs that set the strength of connections between model units. Scrape the entire internet, digitize the world’s books, intellectual property be damned! And so far, it’s been getting the job done. Current LLMs are really powerful, and shockingly capable of executing many (though certainly not all) long, complicated tasks without human oversight.
