Dead-Letter Queues for Webhooks: Safe Replay, Idempotency, and Monitoring

Dead Letter Queues For Webhooks Safe Replay Idempotency And Monitoring

Dead-Letter Queues for Webhooks: Safe Replay, Idempotency, and Monitoring

Webhooks fail. Endpoints time out, databases go down mid-deploy, a TLS cert expires at 2am — and if your system just retries forever or drops the event, you lose data silently. A Dead-Letter Queue (DLQ) is the safety net: instead of vanishing, a failed event gets quarantined for inspection and controlled replay. Here's how DLQs actually work in production, what the monitoring numbers should really be, and how to replay events without causing a second outage.

Idempotency comes before replay, not after

A message can crash after partially succeeding — think "charged the card, then died before marking the invoice paid." If you replay that event blindly, you double-charge the customer. So before any replay logic runs, your consumer needs to check an idempotency key, almost always the provider's event_id, against a record of what's already been processed.