Kubernetes DNS was failing 33% of the time. CoreDNS was fine.

A WordPress site on our platform started throwing Error establishing a database connection. Not always — maybe one page load in three. Reload, and it worked. Reload again, broken.

The obvious suspect is MySQL. It wasn't MySQL. It took us longer than it should have to stop looking there, so here is the whole path, including the two wrong turns.

The first wrong turn: blaming the database

WordPress talks to MySQL through a hostname, not an IP. In our case the pod's config had:

WORDPRESS_DB_HOST = mysql.cdn.svc.cluster.local