Context window

The context window is the maximum amount of text — measured in tokens — that a language model can take into account in a single request. Everything the model knows about the task in front of it must fit: the instructions, the supplied documents, the conversation so far, and the answer it is about to write.

Exceeding it does not produce a graceful degradation. Depending on the implementation, the request fails or the oldest material is silently dropped, which is worse, because the model then answers confidently on the basis of a truncated brief.

Modern windows are large enough that most single-document tasks fit comfortably. The pressure comes from long-running conversations and from applications that stuff in every document they can find rather than retrieving the few that are relevant.