Why ChatGPT Forgets What You Told It Earlier

The model does not remember your last message. It never did. OpenAI’s own documentation is unambiguous: each text generation request is independent and stateless.

What creates the illusion of a conversation is that the entire conversation is sent again, from the beginning, every single time you press enter. The chat window is not a session. It is a document that gets longer, and gets re-read from the top on every turn.

So what is the context window?

It is the ceiling on how big that document can be. OpenAI defines it as “the maximum number of tokens that can be used in a single request”, and notes the number “includes input, output, and reasoning tokens” — so the answer being written counts against the same budget as everything you have already said.

Anthropic’s documentation frames the same limit as capacity rather than arithmetic: the context window is “all the text a language model can reference when generating a response, including the response itself”, and represents “a ‘working memory’ for the model.”

Everything goes in the bundle. Anthropic spells out that “everything in the request counts toward the context window: the system prompt, every message” — including tool results and images. The instructions you never see are in there too, spending your budget.

The forgetting, precisely

Now the mechanism. Each turn adds to the pile — Anthropic calls it “progressive token accumulation”, where each message “accumulates within the context window”.

Eventually the pile exceeds the ceiling. And here is the sentence that explains what you have experienced: chat interfaces “can also manage the context window on a rolling first in, first out basis.”

The oldest messages fall out of the front. Not summarised, not compressed — gone from what gets re-sent. So the model has not forgotten what you said an hour ago; it is no longer being shown what you said an hour ago, and it has no way to know anything is missing.

That is also why the failure is silent. There is no notice, because from the model’s side nothing went wrong.

A bigger window is not automatically better

This is the part that surprises people who go looking for the largest context window they can buy.

Anthropic’s documentation says it directly: “more context isn’t automatically better. As token count grows, accuracy and recall degrade, a phenomenon known as context rot.”

So a long thread degrades in two separate ways at once. The beginning falls off the front, and the middle gets less reliably used even while it is still there. A model with a million-token window will still answer a sprawling conversation worse than a tidy one.

There is a cost consequence too. OpenAI notes that even when chaining requests, “all previous input tokens for responses in the chain are billed as input tokens” — every turn re-bills the whole history. A long conversation gets slower and more expensive with each message, and the last message is the most expensive one you will send.

What to actually do

• Start a new conversation more often than feels natural. A fresh thread is a smaller, cleaner document. Most “the AI got worse” experiences are a thread that should have ended.
• Restate the important constraint when a thread gets long. If something matters, say it again near the end rather than trusting a mention from forty messages ago. Recency is doing real work here.
• Put the task last. The instruction closest to the end is the one least likely to have rotted or fallen off.
• Paste the source rather than referring back to it. “The document I sent earlier” may be nowhere in the request; the document is.
• Do not buy the biggest context window as a substitute for tidying up. Curation beats capacity, and the documentation of the company selling the capacity says so.
• Remember this is separate from memory features. Those store facts outside the conversation, which is why a chatbot can know your name next week and lose the middle of today — and if you would rather it stored nothing, what it does with your chats covers the controls.