Disclosure up front: I work on Radar, the open-source Kubernetes UI whose MCP server is one side of this benchmark. The method is published, so you can check the work.
Everyone is plugging AI agents into Kubernetes right now, and the default move is to give the agent a shell and let it run kubectl. It works. The agent gets there. But nobody seemed to have measured what that approach actually costs, so we built a harness and did.
The setup
52 fault-injection scenarios on a live EKS cluster: crashloops, misconfigurations, resource pressure, broken rollouts, and the messier indirect failures where the symptom is far from the cause.
One model on both sides: Claude Sonnet 4.6.
