Part 12 ended with a hypothesis. Two days later, the hypothesis met data.
The closing line of Part 12 was a guess. Maybe the next bottleneck wasn't more concepts, but the relationships between them. A model can know "dog" and "animal" and "four legs" and still not understand what a dog is. Understanding might live in the connections, not the nodes.
We had a way to test that. Build a sandbox that predicts the next concept that will fire given the current one. Run it on books. If the model can predict that "rock falls" tends to be followed by "ground hits, sound happens," then it's learned something about how the world strings together. If it can't, it hasn't.
I built it that evening. Five books from Project Gutenberg. Twenty-five thousand sentence-to-sentence transitions. Four prediction strategies running side by side, random (the floor), frequency (always guess the most common concepts), cooccurrence (learn which concepts tend to follow which), and retrieval (find similar past sentences and look at what came after them).
The results were not what I wanted.
