The Real Cost of "It Works on My Machine"
Background tasks get a design pass no other code gets. A web endpoint is reviewed, its input is validated, its failure path is traced. A cron job that writes rows to a table is the exception — its default is agreed-on-paper and its failure behavior is an afterthought.
Most Python background work begins life as a tight loop: fetch an id, do the work, mark it done, repeat. That shape is honest about the happy path and dishonest about everything else. When the process dies between "fetched" and "marked done", the loop runs the same record a second time on restart. When a worker reconnects after the database hiccups, it does not know which step committed already, so it replays the whole batch. When a second instance starts because the first one is running slow, both lay claim to the same row. The codebase is exactly correct in the junction where a crash is most likely, which is also the single moment it can least afford to be wrong.
This article argues that the walk-way of disaster is not the crash. It is the scattered. When the question "has this unit been processed" lives across twenty loose branches, every crash demands a fresh re-accounting by hand. The stable fix is to stop treating the task body as the unit of progress and to start treating a single field — the task state — as the unit of truth.
