9 min read · warehouse execution system, global retail & manufacturing
A few years ago I worked on a warehouse execution system (WES) — the software that tells a warehouse what to pick, pack, and ship next, in real time, for retail and manufacturing clients moving somewhere between 5 and 10 million requests a day. Orders come in, inventory gets allocated, pick tasks get generated, shipments get confirmed — and every one of those steps talks to the others through messages.
The system ran on ActiveMQ. It had worked for years. But at that volume, two problems had stopped being rare edge cases and started being a weekly occurrence: duplicate order processing, and failures nobody noticed until a customer did. This post is about what we actually changed — not just "we moved to NATS," but the specific patterns that made the new system trustworthy instead of just newer.
The problem wasn't really the broker
It's tempting to blame ActiveMQ itself. That's not quite fair. The real problem was what had been built around it over the years: a direct, synchronous-feeling coupling between "write to the database" and "publish a message," with no shared transaction between the two. At low volume, that gap almost never showed up. At 5–10M messages a day, it showed up constantly, in two specific ways:
