Most people's first experience with an LLM API is deceptively simple: send a prompt, get a reply. It feels like magic, and for a single question-answer exchange, it basically is. But the moment you try to build something that holds an actual conversation one where the model remembers what you said three messages ago you run into a problem that isn't obvious until you hit it: LLM APIs are stateless. Every request is a blank slate unless you explicitly hand the model its own memory.
That was the core challenge behind a recent project I built during my internship a chatbot backed by a real LLM API ([OpenAI / Gemini]) with genuine multi-turn conversation support, not just a scripted request-response loop.
*The problem nobody mentions upfront
*
You can't just "turn on" memory. Every conversation turn has to be manually tracked and resent with each new API call, which means the developer, not the model, is responsible for deciding what counts as context. And that decision has real consequences: send too little history and the bot forgets things it should remember; send too much, and you run into token limits and rising costs as the conversation grows.
