What is Token Window

Definition

The token window, or context window, is the maximum amount of text, measured in tokens, that a language model can consider at once, encompassing both the input prompt and the generated output, which bounds how much information the model can use in a single interaction.

« Back to Glossary Index

Defines how much context a model can use in one request
Larger windows let models handle long documents and conversations
Guides chunking strategies for feeding large content to models
Affects cost, since more tokens in the window mean higher usage

Real World Example

A team summarising long contracts hits the token window limit, so they split each contract into sections that fit the window, summarise each, then combine, ensuring no part exceeds what the model can read at once.

FAQs

What is a token?

A token is a chunk of text, often a word or word-piece, that the model processes as a unit; token counts determine context usage.

Why does the token window matter?

It limits how much input and output a model can handle at once, shaping how you structure prompts and long documents.

What happens if input exceeds the window?

The model cannot process the overflow, so text must be truncated, chunked, or summarised to fit within the limit.