Nineteen times in six months, a database that was supposed to be boring quietly corrupted itself. Each incident took the same shape: a backup monitor or data pipeline reported an error, PRAGMA integrity_check confirmed corruption, and an on-call engineer stopped a control-plane process, restored from a snapshot, and tried to explain what had happened. Nobody could. The trigger changed every time — different shard, different customer, different time of day. The only constant was that a committed write had somehow stopped existing.
The bug behind all of it had been sitting in SQLite's Write-Ahead Logging code since 2010. It took Tailscale and the SQLite core team months of forensics to find it, one fix release that had to be withdrawn, and two more months of waiting for positive proof that the real fix worked. This is the case study: how a rare data race hides inside one of the most tested codebases in the world, why Tailscale hit it when almost nobody else could, and what the hunt teaches about the difference between standard and non-standard ways of running "boring" technology.
A Database That Corrupted Itself
Tailscale's control plane looks like a single public endpoint, but internally it is a set of coordination servers, or shards. Each shard owns a slice of tailnets and has its own SQLite database, accessed exclusively by one Go process. That single-writer design is exactly how SQLite is meant to be used: one writer, serialisable transactions, no cross-process locking drama.
