A new study evaluating 67 frontier AI models from 21 providers has uncovered what researcher Josef Chen calls the “co-failure ceiling,” a mathematical limit that shows combining multiple AI models doesn’t work nearly as well as companies assume. The gap between expected and actual failure rates runs roughly 2.25 to 2.5 times on standard benchmarks.
The math behind the broken safety net
Chen’s paper, titled “When Does Combining Language Models Help? A Co-Failure Ceiling on Routing, Voting, and Mixture-of-Agents Across 67 Frontier Models,” submitted to arXiv on June 25, 2026, demonstrates why the assumption that multi-model systems cover each other’s blind spots is mathematically flawed. The problem is that models tend to fail on the same questions more often than statistical independence would predict.
The key metric is what Chen calls beta: the rate at which all models in an ensemble fail simultaneously on the same query. On the MATH-500 benchmark, the observed beta was 5.2%, compared to a modeled estimate of just 2.3%. The pattern held across different task types. Execution-graded code benchmarks showed a beta of 7.9%. Free-response questions hit 12.7%—for certain categories of questions, roughly one in eight prompts will stump every model you throw at it, no matter how clever your routing strategy is.
