If you're calling an LLM API for every single user request, you're almost certainly paying for the same question more than once. Not because your users are dumb because human beings ask the same thing in a dozen different ways, and a normal cache only matches exact strings.
"how do I reset my password" and "reset password help pls" are, to a dict or a Redis GET, completely unrelated. To your users, they're the same request. Every mismatch is a full-price model call you didn't need to make.
Let's fix that. By the end of this post you'll have a working semantic cache, understand exactly where it breaks, and know how to fix that too.
The Problem, Concretely
import redis
