Imagine you have built a backend server that handles requests perfectly in development. It easily survives a few hundred users. Then, your application gets picked up on social media, and suddenly 10,000 requests hit your server at the exact same second.
Connections pile up, requests time out, CPU usage spikes, and users are stuck staring at loading screens or a dreaded 502 Bad Gateway error.
Most engineers know the obvious fix: add more servers or put something in front of the backend. But what exactly goes in front? The moment you enter the realm of system design, you hear three terms used interchangeably: Reverse Proxy, Load Balancer, and API Gateway. Even experienced engineers mix them up because they all sit between users and servers. However, they exist for completely different reasons, protect against different failures, and solve distinct scaling problems.
1. The Starting Point: Direct Connection (Layer 0)
In the simplest version of the web, a client sends a request directly to a backend server, and the server sends back a response. This works fine until your production environment starts taking heavy traffic.
