GitHub had an interesting incident last August. A component in Central US failed under load, and when it started recovering, the recovery took much longer than it should have. The culprit: clients hammering the recovering system with retries.
What happened
During the outage, GitHub's auth system was down. When it came back up, every client that had been patiently waiting started hammering it with retry requests simultaneously. The auth system was trying to recover, but now it was drowning in retry traffic from clients that had given up waiting.
This is the retry-loop trap. You've probably seen it in various forms:
A health check endpoint that returns 503, so clients retry every second
