TL;DR
Traditional caching checks if you've seen the exact same input before. Semantic caching checks if you've seen a meaningfully similar input before — using embeddings instead of string matching. For LLM-backed apps, this can cut API costs by 30–70% and slash latency from seconds to milliseconds, without touching your prompts or model choice.
The Problem
If you're building anything on top of an LLM API, you've probably noticed:
Users ask the same question in a dozen different ways ("What's your refund policy?" vs "How do refunds work?" vs "Can I get my money back?")
