Day 13 of building OrderHub in the open. We have a monolith with real persistence, validation, exception handling, OpenAPI docs, and a Redis cache. Today we add the thing that keeps all of that standing up when someone points a firehose at it: a rate limiter.
The problem is simple to state. Without a limit, one buggy retry loop, one scraper, or one genuinely malicious client can fire thousands of requests a second and starve everyone else — or knock your database over. You want a fair ceiling on how fast any single client can call you. The token bucket is the classic way to draw that ceiling, and it's nicer than a plain counter because it allows short bursts while still capping the sustained rate.
Here's the mental model. Imagine a bucket that holds up to capacity tokens and refills them steadily over time. Every request has to spend one token to get in. If the bucket has tokens, the request is allowed and the count drops by one. If it's empty, the request is turned away. Because the bucket can be full when traffic arrives, a client can spend a burst of saved-up tokens all at once — but once it's drained, they can only go as fast as the bucket refills. Capacity is the burst size; the refill rate is the long-run limit. "20 requests per minute per client" is just a bucket of 20 tokens that refills 20 tokens over 60 seconds.
