OpenAI CEO Sam Altman continues to bet on scaling large language models and is pushing back against LLM skeptics. A whole generation of researchers held the field back, he says, because they were too confident about what scaling couldn't do.
Betting against LLMs scaling at this point feels quite misguided to me.
Sam Altman, OpenAI
At Stanford, Altman responded to critics like Yann LeCun, who has called LLMs a dead end. Some people tie their identity to a position and can't let go, even when the data proves them wrong, Altman said. "Twitter trolls" predicting OpenAI's failure for years don't bother him either. World models matter for things like robotics, but the data clearly supports continued scaling. Anthropic CEO Dario Amodei recently made similar remarks.
LLMs have already surpassed human intelligence in some areas, Altman argued. An OpenAI model recently disproved a mathematical conjecture that had stumped smart people for a long time, and mathematicians are now asking what that means for their field. "So clearly, LLMs are capable of figuring out new knowledge," Altman said. For very long-horizon tasks requiring high judgment, though, LLMs "seem much worse than people."
