A single made-up number, 1.9 percent, almost killed an experiment that never ran. Then I found the same lie hiding in my Medicare billing engine and my denial classifier. The fix was the same in all three, and it was not a better guess.
The number that never existed
1.9 percent. That was the baseline conversion rate my ad-experiment engine reported for a test that had never served a single impression. The metrics table was empty. The code reached for a hardcoded default. And 1.9 percent walked out the door wearing a suit.
Here is why that is not just cosmetic. The baseline feeds the formula that decides how long a test has to run before you are allowed to trust it. Feed that formula a made-up baseline and the sample size it asks for can be off by 20 times in either direction. So you either burn weeks on a test that quietly finished long ago, or you crown a winner on noise. The experiment was dead before the first impression, and the dashboard looked healthy the whole time.
An error gets a ticket the same afternoon. A plausible number gets a quarterly review, maybe, eventually. A made-up number does not look made up. That is the whole problem.
