Hello, I'm Shrijith Venkatramana. I'm building git-lrc, an AI code reviewer that runs on every commit. Star Us to help devs discover the project. Do give it a try and share your feedback for improving the product.
Large Language Models keep getting smarter.
But there's a problem: users don't experience intelligence directly. They experience latency.
If a model takes 30 seconds to write an answer instead of 3 seconds, most users won't care that it scored higher on some benchmark.
This creates an interesting engineering challenge:
