Bigger Context Windows Are Not Making AI Answers Better
We went from pages to libraries of context, and the answers did not improve proportionally. Attention is still the scarce resource, and prompt craft is its economics.
Context windows grew a hundredfold while my answers improved maybe twofold, and the gap between those numbers is where most bad AI experiences now live. The marketing implies that more context equals more understanding, but models do not read a million tokens the way you hope; they allocate attention, unevenly, and attention is the actual scarce resource.
My own experiments keep confirming it. A model given forty pages of documentation and one question usually answers better when given the three relevant pages, because retrieval plus a small window beats brute-force inclusion. Long context is not a memory, it is a bigger room with the same flashlight, and what matters is where the flashlight points.
This is why the practical skill has shifted from prompt engineering to context curation. The best results I get now come from deciding what the model does not need to see: strip the boilerplate, cut the tangents, put the crucial constraint at the beginning and repeat it at the end. Curation is unglamorous work that the bigger-window marketing says should be obsolete, and it stubbornly is not.
There are real exceptions, whole-codebase comprehension, hours-long transcripts, and credit where due. But even there, the tools that win are the ones managing attention for you, with retrieval and summaries rather than raw inclusion.
The takeaway is deflating and useful: a bigger window does not absolve you of thinking about what goes in it. The models got a bigger desk; the work still gets done by the person deciding what to put on it, and that person, for now, is you.