The Context Window Lie Almost Cost Me a Real Decision

I bought the big-context pitch for a year. A model that reads your whole drive in one go must be smarter, right? Then I handed one a 200-page contract and asked what I was agreeing to, and it gave me a confident, tidy summary that missed the three clauses that actually mattered.

Here's the lie: context window size gets sold as comprehension. It isn't. A model can hold a million tokens and still fixate on the wrong sentence. The number tells you how much you can paste in, not how well it reasons about what's inside. I'd confused storage for understanding.

What actually moved the needle was narrowing the question. Instead of 'summarize this,' I started asking 'list every clause about liability and quote it.' The same model, the same document, a completely different answer, because I stopped asking it to be broadly smart and started asking it to be narrowly right.

The trap is treating a huge context window as permission to dump everything in and stop thinking. It's a convenience, not a guarantee, and the bigger the window the easier it is to assume the model saw what you meant. It didn't. You still have to point at the thing.

My rule now: small, specific, verifiable questions beat one grand dump. The context window is a parking lot, not a brain, and I stopped mistaking the size of the lot for how well the car drives.