Tokens are an important foundational concept that underlie how we interact with LLMs. @greggyb and myself recorded a quick video to demystify what a token actually is and unpack several other related concepts.
What's in the video
What is a token? Tokens are the basic unit of input and output that LLMs operate on.
What is retrieval-augmented generation (AKA RAG)? Providing a model with extra data that isn't already in the model's training dataset, and is relevant to a specific query.
What are context windows? Context windows are how many tokens a model can use for input and output. Context window size varies from model to model, but it has increased a lot in recent years.
