Across one of our agents, the large majority of genuine defects - eight out of nine, when I went back and classified them - belonged to a single category. Not off-by-one, not a race, not a bad regex.
The category is: the code was written correctly, it was tested correctly, and it never executed on the path that mattered.
Every one of them looked healthy on a dashboard. Every one had passing unit tests. The tests passed because they called the function directly, and the function was fine. The wiring was not.
Case 1: the feature that shipped switched off
We publish forecast intervals, and at some point we added conditioning: instead of reading historical quantiles across all market conditions, filter the sample to periods resembling the current one. It matters a lot - unconditional intervals carry a permanent allowance for turbulence that a calm market has not earned.
