Context windows

It's a desk, not a memory#

The instinct is to imagine the model remembering your conversation the way a person would: things happen, they go into memory, memory is consulted later. That's the wrong picture, and almost every confusing behaviour follows from the wrong picture.

A better one: the model has a desk. Everything it can take into account has to be physically on that desk, laid out flat, at the moment it answers. Not filed nearby. Not retrievable on request. On the desk. The context window is the size of the desk.

Here is the part that surprises people. The model has no memory of your previous messages at all. Each time you hit send, the entire conversation up to that point is re-sent, from scratch, and laid out on the desk again. The model reads the whole thing fresh, answers, and then forgets it completely. The continuity you experience is an illusion maintained by the software around the model, which keeps a transcript and re-sends it every single turn.

So why does it "forget"?#

Because the desk fills up. Once the transcript grows past the window, something has to come off the desk to make room. Products handle this in different ways: some silently drop the oldest messages, some compress earlier parts into a summary, some warn you, and some just start behaving worse without saying anything.

That's why the classic failure looks the way it does. You have a long, productive session. You established an important constraint near the beginning. Forty messages later the model violates it, and when you point this out it apologises and agrees, sincerely, because you have just put the constraint back on the desk. It never forgot in the human sense. The instruction had simply been carried off the far edge.

Output shares the desk too

Input and output come out of the same budget. A long document you paste in does not just cost you space, it eats into the room available for the reply. This is why asking for an exhaustive analysis of an enormous file can produce an answer that stops mid-sentence. There's a separate, smaller cap on how long a single reply can be as well, which is the max output column on Model facts.

How big is it, in real terms?#

Windows are measured in tokens, which are chunks of text a bit shorter than a word. To make that concrete: Anthropic gives a one-million-token window on its current flagship models as roughly 555,000 words, or about 2.5 million characters. That's genuinely enormous, somewhere in the region of five or six long novels.

Note the qualifier, because it is the whole lesson of Tokens in one line: the same one-million-token window on the previous generation of models is given as roughly 750,000 words. Same window, same vendor, same page — different tokeniser. A window's size in tokens is a fact about the product; its size in words is a fact about the tokeniser, and it changes underneath you.

Current windows sit between a couple of hundred thousand tokens and a million, and that range keeps moving, which is exactly why the numbers live on their own dated page rather than in this paragraph.

Why bigger is not automatically better#

A million-token window sounds like it makes this whole concern obsolete. It doesn't, for three reasons.

1. You pay for the whole desk, every turn

Since the entire transcript is re-sent on every message, cost scales with conversation length in a way people find counter-intuitive. Message fifty in a long session costs far more than message one, because you're paying to re-read everything each time. A large window is permission to spend a lot of money quickly, and the single biggest lever against this is caching, which is why it's on the concept list.

2. Being in the window is not the same as being read carefully

Having something in the window is not the same as the model weighing it properly. The mechanism that does the weighing is called attention, and it thins out as the material stretches. Two separate findings sit behind that, and they are worth keeping apart, because one is about where material sits and the other is about how much of it there is.

Position matters. The paper that named this measured accuracy as the same relevant passage was moved through the context, and found performance highest when it sat at the beginning or the end and markedly worse when it sat in the middle. Accuracy is highest at both ends and worst in the middle — the shape is a U, which is not what anyone expects from a system advertised as reading everything you give it.

Length matters too, and it survives better measurement. That first result is from 2023 and the models it tested are long retired, so on its own it would be exactly the kind of finding this site warns you about. What keeps it alive is that later benchmarks, built specifically to remove the shortcut of matching words between the question and the answer, still find accuracy falling substantially as the context grows, on models sold on the strength of their enormous windows. Making the retrieval require a small inferential step rather than a keyword match is enough to expose it.

The practical reading is unchanged and now better supported. Filling a huge window to the brim and assuming everything in it carries equal force is a mistake. Relevance beats volume: ten pages of the right document reliably outperforms three hundred pages containing it.

3. Long contexts are slower

More to read means more time before the first word comes back. On a long session this becomes noticeable, and on anything interactive it matters.

What to actually do about it#

Start a new conversation more often than feels natural. People treat a long thread as accumulated value. Frequently it's accumulated noise that is costing you money and diluting attention. When you change tasks, start fresh and paste in just the conclusions that still matter.

Put the important thing near the end. If you have one critical constraint and a pile of background, the constraint goes last, closest to the question. Recency is doing real work here.

Paste the actual material. Describing a document from memory and asking the model to work from your description is strictly worse than putting the document on the desk. This is the highest-return habit change for most people, and it is entirely a context-window insight.

Re-state constraints in long sessions. Not because the model is being difficult, but because you're managing a desk. Restating is cheap; a violated constraint discovered late is not.

Treat "it forgot" as diagnostic. When a long session starts producing worse work, contradicting earlier decisions, or losing the plot, that's your signal, and the fix is almost never a better prompt. It's a fresh session with the relevant context carried over deliberately.

Sources#

The 555,000-word figure for a one-million-token window, and the 750,000-word figure for the previous generation, are both Anthropic's own, from its models overview, where they appear as tooltips on each model's context window. Re-read 14 August 2026.

The position effect comes from Lost in the Middle: How Language Models Use Long Contexts (Liu, Lin, Hewitt, Paranjape, Bevilacqua, Petroni and Liang, first posted July 2023, published in Transactions of the Association for Computational Linguistics), which tested multi-document question answering and key-value retrieval and reports that performance "is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models."

That the problem persists on newer models comes from NoLiMa: Long-Context Evaluation Beyond Literal Matching (Modarressi, Deilamsalehy, Dernoncourt, Bui, Rossi, Yoon and Schütze, February 2025, ICML 2025), which rebuilds the standard needle-in-a-haystack test so that the question and the buried answer share almost no wording, forcing the model to infer the link rather than pattern-match it. Under that condition performance falls sharply as the context lengthens, including on models with very large advertised windows. Its per-model figures are deliberately not reproduced here: they are measurements of specific 2025 models, and this page keeps figures on one page for exactly that reason. Both papers read 14 August 2026.