Originally published on software-engineer-blog.com.
Ask an AI the same question twice. Get two different answers. That's not a glitch you tolerate — it's the entire mechanism working exactly as designed.
Start below the buzzword: a language model never sees a finished sentence. It only ever answers one tiny question, over and over — given everything written so far, what's the next chunk of text?
One-line mental model: the model outputs a probability over every possible next token → it samples from that distribution instead of always grabbing the top score → the winning token gets glued onto the text → the exact same question runs again from scratch, one token at a time.
The concrete example: finishing one sentence
