When I built the ETL pipelines for three programmatic directory sites in April — Top AI Tools (HuggingFace data), Find Games Like (Steam data), and Open Alternative To (GitHub data) — I had to figure out rate limits for three completely different APIs in the same week. The numbers, the failure modes, and the right way to handle errors are all different.
Here's what I actually shipped and the reasoning behind each number.
Steam: 250ms, deliberately aggressive
Steam's developer docs are sparse on hard rate-limit specifics. What I found from community discussion and trial: roughly 200 requests per 5 minutes per IP on the public Web API, which works out to one request per 1.5 seconds as a documented-safe interval. My code comments this openly:
await sleep(250); // Steam rate limit: ~200/5min, 1.5s is safe; 250ms is aggressive but usually fine
