When a suite that calls a model goes red, it usually goes red in clusters: one rate limit, one bad deploy, one model alias moving, and forty tests fail at once. Reading that as forty problems is the default behaviour of every test report, and it is wrong in a way that costs hours.
Why grouping by test name fails
A test report is organised by the thing the runner knows: file, class, test name. That organisation assumes failures are independent, which is a decent assumption for unit tests and a bad one for tests that all go through the same HTTP client to the same provider. When the shared dependency wobbles, every test that touches it fails, and the report presents them as unrelated because their names are unrelated.
The consequence is not just wasted reading. It distorts every downstream number: your flake counts spike for forty tests that are individually fine, your triage backlog fills with duplicates, and a genuine single-test regression that happened during the same run is buried in the noise. Grouping by cause is what makes the run legible — one incident, one row, forty affected tests, and any test that failed for a different reason standing out immediately.
What goes into a signature
