Message Queues Explained: Why LinkedIn Built Kafka and Changed Async Communication Forever

Covers: Point-to-Point vs Pub-Sub, Kafka Internals, Delivery Guarantees, Dead Letter Queues, Backpressure

The Upload That Broke Everything

In 2011, LinkedIn's activity feed was choking. Every time a user updated their profile, viewed a connection, or clicked an article, the system needed to:

Update the activity feed