This comment makes so much sense relative to what I've seen with Claude's 1M context window. It reliably fails to succeed a task with a prompt where I just stuff in a big blob of data in the middle as context. But when I use emebddings to only select a small relevant subset of that data, it always passes the task.
Misremembered, the main thrust of the comment still stands, the 100K context window isn't "real", it would be absurdly expensive to do it for real. They are using a lot of approximation tricks to get there.
Yes, that's the point now for competing AI research-for-profit companies, whatever metric is technical and sounds important, is going to be used in marketing and valuation determinations. It will be explored for research I'm sure, and then determine its product viability. It's nice competition, but agree, that it can be deceptive.
