A million tokens of context is not memory, and the confusion is costing people money

This year's model launches competed on context windows — a million tokens here, 1.05 million there. The marketing implies a promise: the model can hold everything. My colleague took the promise literally, uploaded his entire project archive into one chat, and started getting worse answers than before. He thought the model got dumber. It got distracted.

A context window is working memory, not a filing system. Everything in it competes for attention on every turn. Paste in a year of documents and your actual question is a whisper in a stadium. The model doesn't 'know' your archive — it reads all of it, every time, with the same attention budget, and relevance gets diluted by volume. More context isn't more knowledge; it's more noise per answer.

The practical fix is what people used to do before they trusted machines to remember: curation. The best answers I get come from chats where I've included the three relevant documents, not the three hundred. Retrieval — the model or tool finding the right pages — beats dumping, every test I've run. That's what the various 'knowledge base' features actually do, and why they outperform the paste-everything approach.

There's a cost angle that should kill the habit on its own. Tokens in the context are tokens you pay for, on every single turn of the conversation. My colleague's mega-chat was burning his usage cap re-reading his whole archive to answer questions about one file. Efficiency and quality pointed the same direction: smaller.

The mental model that's served me: treat the context window like a desk, not a warehouse. You work well with ten relevant papers on the desk. You work badly standing in a warehouse. The warehouses are great — keep your archive somewhere searchable. Just bring the right folder to the desk when you actually need to think.