1. What Is the Thundering Herd Problem?
The thundering herd problem (also called a cache stampede or dogpile effect) happens when a popular cache key expires or is missing, and a large number of concurrent requests all miss the cache at the same instant. Instead of one request hitting the database, hundreds or thousands do — simultaneously.
Picture a hot product page on an e-commerce site. The cache entry for product:12345 expires at exactly midnight. If that product gets 2,000 requests/second, all 2,000 requests in that window see a cache miss and stampede toward the database at once. The DB, which was only ever meant to serve the occasional cache-refill query, gets hammered with duplicate work — often enough to tip over under load, which then causes more timeouts, more retries, and a feedback loop that can take down the whole service.
The core issue isn't that the cache missed — misses are normal. The issue is redundant, uncoordinated work: every request tries to regenerate the same value independently, when only one regeneration was ever necessary.
2. The Solution Strategy: Locking (a.k.a. "Cache Mutex" / Request Coalescing)
