One event happens. A user gets mentioned in a comment. Now you might need to send a push notification, an email, and an in-app badge, and you have to respect that this person muted email but still wants push. Multiply that by millions of events per hour and the naive version falls apart fast. This is one of those systems that looks trivial from the outside and turns into a mess the moment you try to make it reliable.
The core problem
A notification system is a fanout engine with three hard requirements pulling against each other: deliver fast, never lose a message, and never spam the user. The trouble is that the producers of events (comments, follows, payments, alerts) have no idea how a given user wants to be reached, and the delivery channels (APNs, FCM, an email provider, an SMS gateway) each fail in their own way and at their own speed. If you couple event producers directly to channel delivery, one slow provider stalls the whole thing.
So the first decision is to decouple. Producers write an event. A separate pipeline decides who gets notified and how.
Key design decisions
