Hello, I'm Shrijith Venkatramana. I'm building git-lrc, an AI code reviewer that runs on every commit. Star Us to help devs discover the project. Do give it a try and share your feedback for improving the product.
Large language models keep getting larger.
Hundreds of billions of parameters. Trillions of parameters. Yet somehow, many of these models remain surprisingly fast and affordable to run.
How?
The trick is that most modern frontier models don't use all of their parameters for every token.
