Prominent programmer and hacker George Hotz warns that AI agents in software development do more harm than good. He says he's now in the "LeCun/Marcus camp," referring to AI researchers Yann LeCun and Gary Marcus, who doubt LLMs will ever become truly intelligent.
In his blog post "The Eternal Sloptember," Hotz argues that using AI agents in software development will become one of the industry's most expensive mistakes. He spent six months testing various models and tools, including work on tinygrad. His takeaway is that LLMs deliver fast prototypes but fall apart on the fine details.
Large organizations are especially at risk, he says, because weaker developers can't spot the flawed output. Hotz believes today's language models will never truly be able to code and that world models are needed instead. LLMs are "sophisticated statistical models" designed to "mimic the distribution of programming."
The output is flawed, but in a way that's "harder and harder to detect," exactly what you'd expect from an increasingly accurate statistical model, Hotz says. Quality indicators like syntax and grammar have become useless, he argues, since AI-generated artifacts don't emerge through the same process as human ones. As an example, he cites models that simply comment out a failing test and then report that all tests passed.
