A few weeks ago I published a write-up on inverting the control flow of a settlement API: instead of the endpoint owning window semantics (batching, cutoffs, retries), it became a pure write-and-forget operation, and all of that coordination logic moved to a cold-path scheduler. One of the best responses I got was from a fellow engineer who pushed on the exact point that matters most in any event-driven settlement system: what happens to the message between the moment your transaction commits and the moment the broker acknowledges it?
That question is worth a full article on its own, because the answer we landed on — a two-layer idempotency model instead of a transactional outbox is a trade-off I think gets under-discussed. Most write-ups on reliable event publishing jump straight to "use an outbox," treat it as a solved problem, and move on. It's a great pattern. It's also not free, and for a meaningful class of systems, it's more machinery than the failure mode actually justifies.
The gap everyone is trying to close
The problem is well known: you commit a database transaction, and then you need to tell the world about it via Kafka. If you publish inside the transaction, you risk publishing an event for work that later rolls back. If you publish after the transaction commits, in something like Spring's afterCommit(), you close that hole, but you open a new one: the process can crash, or the network can fail, in the narrow window between the commit finishing and the broker acknowledging the message. Now your database says the work happened, but no one downstream knows about it.
