When you call an external API, things go fine until they don't. A network blip, a server restart, a rate limit. So you add a retry, and most of the time it helps. The problem is that the obvious retry, the one we all write first, can quietly make things worse: it can resend a payment twice, or keep a struggling server down longer than the original failure would have.
In this post we'll build a retry client in Go from the naive version up, fix each thing that bites, and end somewhere a little uncomfortable: telling you to use a library instead. By the end you'll understand exactly what that library is doing, which is the real reason to read this.
The retry everyone writes first
Three tries, wait a second, give up. It looks reasonable, and there are three problems hiding in it.
First, the body. An HTTP client reads the request body to the end when it sends it, and it doesn't rewind. So on the second attempt there's nothing left to send, the server gets an empty POST, and it returns 400. Now you're retrying a 400 that you caused.
