When you ask a chatbot for a pasta recipe, it doesn’t understand “pasta” the way you do. It sees a list of numbers. Those numbers are not random. They are arranged so that “pasta” sits close to “spaghetti” and far from “car.” This is an embedding. It turns meaning into coordinates that a machine can measure, compare, and search. Every time a chatbot finds a relevant answer, recommends a product, or remembers a fact from earlier in the conversation, embeddings are doing the heavy lifting behind the scenes.
Here is the surprising part. The machine never learns what “pasta” means. It only learns which words appear in the same kinds of sentences. That statistical shadow turns out to be so rich that it captures everything from synonyms to analogies. The rest of this article unpacks how that happens, step by step, using the same chatbot interactions you already know from this series.
What exactly is an embedding?
An embedding is a list of numbers that represents something discrete, a word, a token, a user ID, a product. Each number is a coordinate in a high-dimensional space. If you pick the right coordinates, similar things end up near each other. That is the entire idea.
Think of a map. A city’s latitude and longitude don’t tell you its name or history. But if you know that Paris is at (48.9, 2.3) and Lyon is at (45.8, 4.8), you can measure the distance and see they are both in France. Embeddings work the same way, except they use hundreds of dimensions instead of two. Every dimension captures some latent feature of the input, learned from data. The model never labels those features. It just arranges points so that words that behave similarly in text end up with similar coordinates.
