This is just an opinion of what I experience and am witnessing, but looking at how LLMs scale feels like I've seen it before: with CPUs trying to outrun Moore's Law and break the rules of physics.
Heat, power leakage, and diminishing returns made it increasingly expensive to squeeze out even small gains in clock speed. The GHz race shifted because it had to.
For LLMs, more compute, more data, more parameters, and everything just keeps getting better? That curve seems to hit a ceiling and innovation needs to succeed the scaling race now.
History does not repeat itself, but it rhymes. What learnings can we make from history to "predict" a potential future?
History
