I've been hearing about embeddings for a while now, and even as someone who's very conversant with using LLMs as a daily driver and for integrating into smart systems, I wasn't really sure what exactly embeddings were and how they connected with everything else.
In this writeup, I'll be unpacking some of the things I've been able to learn about embeddings — what they are and how to use them as a software developer/engineer.
Turning Meanings into Coordinates
Think of embeddings as turning meanings into coordinates. LLMs are not built to — and cannot — understand words the same way humans do, so they convert text into lists of numbers that represent meaning.
Take the word "dog" for example. An LLM wouldn't straightforwardly understand what the word means until it converts it into a group of numbers:
