Searching billions of documents for a phrase and getting ranked results in tens of milliseconds looks like magic. It is not. It comes down to two ideas working together: an index that maps words to documents instead of scanning documents for words, and a way to spread that index across machines so each holds only a slice. Understand both and full-text search stops being mysterious.
A visual from the full interactive lesson on systemdesign.academy.
The core problem
A database scans rows. If you ask a plain database to find every document containing a word, it reads documents and checks them, which is linear in the amount of data. That is fine for exact key lookups and hopeless for free-text search across huge corpora. You need the opposite mapping. Instead of "given a document, what words does it have", you want "given a word, which documents have it". That inversion is the whole trick.
The second problem is size. One machine cannot hold the index for billions of documents, and one machine cannot serve the query load. So the index has to be split across nodes, and a query has to find the right nodes and combine their answers.
