Why Long Chats Get Confused

📖 Resources & more for this episode: https://unrsnbl.ai/notes/e020-why-long-chats-get-confused AI memory explained: why the LLM doesn't remember anything - and how RAG solves the problem for large documents. When you use a chatbot like ChatGPT, it feels like it remembers what you said earlier. But something important is happening behind the scenes. The chatbot you see is actually a larger system. Inside it sits the LLM - the language model. The LLM itself does not remember anything at all. It's not becoming smarter or retraining itself as you chat. Every single response starts from the same trained model, frozen at the moment training ended. What the system does instead: each time you send a message, it assembles a large prompt - including the conversation history, system instructions, sometimes the current date, tool results, search results, or retrieved documents - and sends the whole thing to the model again. The model processes it and generates a reply. It is not remembering. It is re-reading. This also explains why long chats behave strangely: the context window is limited. When the conversation grows too long, earlier content gets compressed or removed. Once it disappears from the context window, the model cannot see it anymore. For large documents or knowledge bases, a different strategy is needed: RAG (retrieval augmented generation). Instead of stuffing everything into the prompt, the system searches for the relevant pieces and adds only those to the prompt at the moment they're needed. ▶️ Full playlist: https://www.youtube.com/playlist?list=PL3pL28ov_GlKZ8fgcP04yi_nBuBc_i65C 📦 Join us in Telegram: https://t.me/unreasonableai Start tagging your content to indicate this is generated by Human (or not?). More details here: www.contentags.com #ai #shorts #notesonai #aibasics #llm #genai #CTHuman