It was a Tuesday afternoon. Nothing felt different about the deploy. Same process as always. Green tests. Approved PR. Merged to main. Deployed.
Seventeen minutes later the alerts started.
What Happened
We had a background job that processed orders and updated inventory counts across our marketplace. The job ran every few minutes and had been running reliably for months without a single issue.
I had made what I genuinely believed was a minor change. A refactor. Cleaner logic, same behavior. I had tested it locally. The tests passed. A colleague reviewed it and approved it. Everything looked fine.
