When I first started managing microservices at scale, Kubernetes felt like magic — until it didn’t. Pods restarted, requests vanished into the void, and tracing cross-namespace traffic became a weekend sport.

At first, the networking model seemed almost too simple.

A Pod gets an IP. A Kubernetes Service gives that Pod a stable virtual endpoint. CoreDNS handles service discovery. The CNI provides pod-to-pod connectivity. kube-proxy handles service traffic using mechanisms such as iptables or IPVS.

For a small cluster, that model works remarkably well. Then you have 150 services. Then 500.

Then teams start deploying independently, services communicate across namespaces, traffic crosses availability zones, a few workloads become extremely chatty, and somebody introduces three layers of retries because the network is sometimes flaky.