Context Window

The context window is the fixed span of tokens a language model can attend to in a single pass. It includes the system instructions, the user prompt, any retrieved or pasted material, and the tokens the model generates in response. Once the limit is reached, older content must be dropped, summarized, or excluded.

How It Works

A model's architecture defines a maximum sequence length, often ranging from a few thousand to hundreds of thousands of tokens or more. Within that limit the attention mechanism lets each token reference any other, which is what gives the model access to the full window. Because attention cost grows with sequence length, larger windows demand more memory and compute, and quality can still degrade for information buried in the middle of very long inputs.

Why It Matters

The context window bounds how much information a model can use at once, shaping prompt design, document handling, and conversation memory. Applications that exceed the limit use techniques such as chunking, summarization, and retrieval-augmented generation to feed only the most relevant tokens. Managing the window carefully also controls cost, since every token in the window is billed.

Related Terms

The window is measured in token units and processed by the attention-mechanism; retrieval-augmented-generation helps fit relevant data within it during inference.