Sequential Testing and the SPRT: How to Stop a Test Early Without Cheating
Meta description: Peeking at a fixed-sample A/B test inflates false positives. Sequential testing lets you check results repeatedly and stop early without cheating.
TL;DR
Fixed-sample testing assumes you'll wait for a pre-calculated sample size before looking at results. Checking early and stopping the moment you see significance — "peeking" — quietly inflates your real false-positive rate, often far above the 5% you think you're getting.
Abraham Wald's Sequential Probability Ratio Test (SPRT), developed for wartime quality control, is the mathematically rigorous alternative: a procedure built to be checked repeatedly, with pre-calculated boundaries that keep the false-positive rate honest by construction.
