KI-Pioneer Sutton calls synthetic data a "big mistake" in the face of an infinitely complex world

Turing Award winner Richard Sutton calls synthetic data a "big mistake" for scaling large language models. The world is infinitely complex, and any simulation of it is "microscopic," with human expertise acting as a bottleneck that blocks real scaling. Sutton's alternative is agents that learn continually from their own experience instead of relying on frozen models.

Raccontata dathe-decoder.com