Building Production-Grade LLM Applications
A short note, just sharing some experience that might be useful for those who are starting to explore building LLM applications. A few things I've come to understand that have become important notes for me.
1. LLMs are stateless.
Don't assume that once you've successfully integrated with an LLM vendor's API, you automatically get the same experience as interacting with ChatGPT directly. ChatGPT is a product; what we call through the API is a model — and the model has no memory at all.
You say "my name is Iwan" in turn 1? Don't expect the LLM to remember it in turn 2. What actually happens: the entire conversation history is re-sent with every request. So "memory management" isn't the model's job — it's ours :D
