You've probably seen the memes. Someone asks GPT-4 how many r's are in the word strawberry, and it confidently answers 2.
It's not a reasoning failure. It's not even a knowledge gap. It's a direct consequence of how every modern LLM reads text — and once you understand it, a whole category of weird AI behavior suddenly makes sense.
For Day 3 of our interactive system series, we built a hands-on BPE tokenizer simulator. You type into a real tokenizer engine, watch tokens form and merge in real time, and then complete three escalating challenges that expose the cracks in the system.
Before transformers can process text, it needs to be converted into numbers. That's the tokenizer's job. But naively assigning one number per letter is wildly inefficient — English has 26 letters, but the real vocabulary of the web is enormous.
Byte-Pair Encoding is the compression algorithm that solves this. Here's how it works:
