How do Large Language Models (LLMs) maintain memory during a conversation?
Large Language Models (LLMs) don't have true "memory" in the human sense across conversations. Each interaction is largely stateless; the model treats every new prompt as if it's the first. However, during an ongoing conversation, what appears as memory is handled through context windows. The LLM re-receives the entire chat history, including its own previous responses and your inputs, appended to the current prompt. This allows the model to "remember" earlier parts of the discussion because that information is literally fed back into its input. The length of this re-fed history is limited by the model's context window size, measured in tokens. Once the conversation exceeds the context window, the oldest parts of the history are dropped, causing the model to "forget" them.