A worksheet in prod stopped saving.
The pod was healthy. 404 MiB of a 2 GiB limit, 655m of 1500m, no restarts. I didn't believe that, so I went and looked at the database too. Three active queries cluster-wide, 12% CPU, all three nodes live. Idle.
Nothing was exhausted, nothing had crashed, and the service still couldn't write.
The software
It's a collaborative editor. Teachers build worksheets, whiteboards and lesson plans, and several people can have the same document open at once. Every document is a CRDT, built on Loro. The browser holds a replica and applies edits to it locally, then pushes them over a WebSocket to a sync server. The server keeps its own copy of each open document in memory, merges whatever arrives into it, and writes the result to CockroachDB v25.x.
