I Replaced kube-proxy with eBPF in Production (And Why My Monitoring Went Blind for 6 Hours)

I Replaced kube-proxy with eBPF in Production (And Why My Monitoring Went Blind for 6 Hours)

From a "simple" Cilium upgrade to a 6-hour outage where my SIEM couldn't see a single packet — the real story of deleting kube-proxy, the eBPF program that saved me, and why the docs never warned me about the one metric that matters.

The Setup

It started with a cilium upgrade command I ran on a Tuesday evening.

My homelab cluster — the same 4-node bare-metal setup I wrote about last month (Dell OptiPlex + 3 Raspberry Pis, Talos Linux, Cilium, ArgoCD, Longhorn) — was running Cilium 1.15 in kube-proxy-replacement partial mode. Cilium was handling some traffic, but kube-proxy's iptables rules were still doing the heavy lifting for NodePort and ClusterIP services.