In 2013 four researchers went back to a completed mammography study and asked it a question it had not been designed to answer.
The original study had put 50 radiologists in front of 180 mammograms, twice. Once unaided, once with computer-aided detection marking suspicious regions. The finding was a null. On average, computer aid changed nothing measurable, and the profession moved on.
Andrey Povyakalo and his colleagues at City University London reanalysed the data by splitting the readers instead of pooling them. What they found was that the tool had done two large things at once, in opposite directions. For the 44 least discriminating radiologists, on 45 relatively easy cancers, computer aid was associated with "a 0.016 increase in sensitivity (95% confidence interval [CI], 0.003-0.028)". For the 6 most discriminating radiologists, on the 15 hardest cancers, "with CAD, sensitivity decreased by 0.145 (95% CI, 0.034-0.257)".
The weakest readers got slightly better at the cases that were already easy. The best readers got substantially worse at the cases that were hard, which is to say the cases where a radiologist is the only thing standing between a patient and a missed cancer. Averaged together, those two effects cancelled, and the study reported that nothing had happened.
