Great Stack to Doesn't Work #4 — Kubernetes: "Pod Is Running, App Is Dead"

A survival guide for when everything goes wrong in production.

The pod is Running. STATUS says Running. kubectl says Running. The deployment shows 3/3 replicas available. Every signal says this thing is alive.

But your users are getting timeouts. The health check endpoint returns 200, but the application thread pool is exhausted. The container is up. The process is running. The application is dead.

Kubernetes trusts your probes. If your probes lie, Kubernetes believes the lie.

The Three Probes: liveness, readiness, startup