Growthtrait
All terms
LLM Context Window

Context Window

A context window is the maximum amount of information, measured in tokens, that a large language model can take into account in a single request. It includes the instructions, the conversation so far, any documents or tool results, and the answer the model writes. Anything outside the window, the model simply cannot see.

Think of the context window as the model's working memory for one task. Everything it needs to answer well has to fit inside it at the same time. When a chat grows too long or a document is too large, older or extra content has to be dropped, summarized or retrieved only when needed.

How Big Are Context Windows?

Context windows have grown fast. Early chat models handled a few thousand tokens. Frontier models in 2026 handle around a million: as covered in our GPT-6 Astra vs Claude Fable 5.1 comparison, Astra supports roughly 1.05 million tokens and Fable 5.1 about 1 million, with output capped around 128,000 tokens. A million tokens is roughly 750,000 English words.

What Fills a Context Window?

  • The system prompt and instructions.
  • The conversation history.
  • Documents, search results or files pulled in for the task.
  • Tool definitions and tool outputs, which can be surprisingly large in agent setups.
  • The model's own response, including any reasoning tokens.

Is a Bigger Context Window Always Better?

No. Quality tends to drop well before the window is full: models can miss details buried in the middle of a long input, and every extra token adds cost and latency. That is why context engineering, choosing exactly what goes into the window for each step, matters more than raw size. Our guide to context engineering covers the practical strategies, including giving each sub agent its own clean context in a multi-agent system.

Context Window vs RAG

A large window lets you paste more in, but it does not decide what is relevant. RAG does that: it searches a knowledge base, usually with embeddings stored in a vector database, and puts only the most relevant passages into the window. Most production systems use both.

Frequently asked questions

What is a context window in AI?+

A context window is the maximum number of tokens an AI model can consider in one request, including your instructions, conversation history, documents and its own answer.

What happens when you exceed the context window?+

The request fails or the oldest content is cut off, so the model no longer sees it. Applications usually summarize history or retrieve only relevant passages to stay inside the limit.

Does a bigger context window mean better answers?+

Not necessarily. Models often get less reliable with very long inputs, and more tokens cost more. Sending focused, relevant context usually beats sending everything.

Related terms

Token

AI Token (LLM Token)

A token is the basic unit of text a large language model reads and writes. Before a model processes a prompt, it splits the text into tokens, which can be whole words, parts of words, punctuation or spaces. Tokens are how models measure input length, output length, context limits and price.

Context Engineering

Context Engineering

Context engineering is the practice of curating and maintaining the right set of information an AI model sees at each step of a task, rather than only phrasing a single prompt. It covers the whole information environment around the model, including instructions, retrieved documents, memory, and tools.

RAG

Retrieval-Augmented Generation

RAG (Retrieval-Augmented Generation) is a technique that lets a large language model retrieve relevant information from an external source before generating an answer, rather than relying only on what it learned during training. It grounds the model's response in specific, up-to-date documents.