Kubernetes Probes as Failure Detectors: Semantics, Timing, and Cascading Risk
Kubernetes probes are distributed-systems failure detectors with different control effects. A readiness failure removes a Pod from Service endpoints; a liveness failure asks the kubelet to restart the container. The distinction matters because detection is necessarily imperfect: aggressive thresholds reduce detection latency but increase false positives under transient load.
Readiness protects traffic; liveness repairs deadlock
Readiness should answer: “Can this replica safely accept new work now?” It may include critical local state and indispensable downstream dependencies, but indiscriminately probing every dependency can create a cascade: one database slowdown marks every replica unready, eliminating all capacity precisely when graceful degradation is needed.
Liveness should answer a narrower question: “Is the process irrecoverably stuck such that restart is the best available remediation?” It should not fail because a remote dependency is unavailable. Restarting healthy processes during a network partition adds cold-start pressure without repairing the dependency.
