Six services sit between a buyer clicking "buy" and a vendor eventually getting paid: auth, gateway, catalog, inventory, order, and now settlement. Any one of them can be slow, restarting, or fully down at any moment — that's the normal condition of a distributed system, not an incident you fix once and move past. The question worth writing down isn't "how do we prevent that." You can't, fully. It's: when it happens, which requests should degrade to a worse-but-safe answer, and which must refuse outright rather than guess?
The instinct most people reach for first is a single global rule — either "always degrade gracefully" or "always fail fast." Both are wrong the moment you apply them uniformly across services that don't carry the same kind of risk. I found the actual criterion isn't "how important is this service." It's whether the business cost of guessing wrong is reversible.
The criterion, not the checklist
A service whose worst-case wrong guess is "the buyer sees something slightly stale" may degrade — the guess self-corrects the moment the dependency comes back, and nothing irreversible happened while it was wrong. A service whose worst-case wrong guess is "we sold something we don't have" or "money moved that shouldn't have" must fail fast, because there is no UPDATE statement that un-ships a package or un-spends a vendor's payout. That's the whole rule. Everything below is just applying it, service by service, to real code that's actually running.
