Every few months someone forwards me a screenshot: a detector says their essay is "98% AI-generated," and they wrote every word themselves. Usually they're a non-native English speaker. I've spent enough time inside these systems to explain why that happens, and why the number on the dial is not a probability in any sense you'd want to bet on.
Blunt version up front: statistical AI-text detection is a real technique with a real signal, and it is also fundamentally incapable of the thing people use it for — adjudicating individual documents.
What the classifiers are actually measuring
Almost every zero-shot detector reduces to one idea: run the candidate text through a language model and ask how surprised the model is. The core quantity is per-token log-likelihood:
import math
