Context window
The maximum number of tokens an LLM can process at once — prompt plus answer. Everything the model "sees" must fit here. It has grown explosively: from ~2k tokens (GPT-3) to hundreds of thousands or millions in modern models.
Why it matters: the context window determines how much document, history, and instructions you can give a model at once. A larger window lets you insert whole books, codebases, or long RAG results. But a "large window" isn't free: the cost and latency grow (often quadratically with attention), and models suffer from "lost in the middle" — they remember the beginning and end better than the middle of a long context. "Needle in a haystack" tests measure how well a model finds a specific detail in a long context. That's why RAG (fetch only what's relevant) is often better than just filling the window. Related to context-length extension and RAG.