My Backup Leaked 156 GB, Filled the Disk, and Broke Itself. The Fix Was One Letter.

This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry.

My nightly backup has a verify step. It restores the fresh dump into a throwaway postgres container and runs sanity queries — because a backup you never restored is a hope, not a backup.

One week it started flapping. postgres did not become ready. That was the whole error. The dump itself succeeded. The storage tarball succeeded. Only the gate failed — the step whose entire job is to tell me the backup is real.

Then one morning the box stopped answering SSH.

The innocent suspect