Practices
Context window
definition
A context window is the maximum number of tokens a language model can process in a single request, covering the system prompt, conversation history, tool definitions, retrieved content, and the response.
In depth
Everything the model knows during a request lives in the context window. For an agent that means the system prompt, always-loaded rule files, the definition of every available tool, the conversation so far, the full text of every file read and every tool result returned, and the reply being generated. It is a single shared budget, and agents consume it far faster than chat does — one large file read or one verbose tool response can spend more of it than an entire human conversation.
Larger windows have not made the constraint go away, for two reasons. Attention quality degrades unevenly across a long context, with material in the middle attended to less reliably than material at the edges — the effect commonly described as 'lost in the middle'. And cost and latency scale with tokens processed, so a design that fills a large window on every turn is expensive per turn regardless of whether it is more accurate.
The practical techniques are all forms of spending the budget deliberately. Progressive disclosure loads a summary and fetches detail only when needed — the mechanism behind Agent Skills. Compaction summarises earlier turns to reclaim room. Sub-agents run a task in a separate window and return only a conclusion. Prompt caching makes a stable prefix cheap to re-send, which is why putting invariant instructions first is a cost decision as much as a clarity one.
The failure mode teams notice too late is context pollution: a rule file that grew for a year, forty installed tools whose definitions are always present, a stale summary that contradicts the current task. The model does not tell you it is confused. It just gets quietly worse while costing more.
Why it matters for governance
The context window is also the attack surface, and that framing is more useful than the cost one. Every token in it was placed there by something — a rule file, a tool description, a fetched page, a tool result — and the model cannot reliably distinguish which of those sources deserves authority. Knowing what occupies the window, and which artifact version put it there, is what makes an injection incident reconstructible instead of theoretical.
Frequently asked
- Does a bigger context window remove the need to manage context?
- No. Retrieval accuracy degrades across long contexts, and cost and latency scale with tokens processed. A large window raises the ceiling; it does not make filling it a good idea.
- What consumes an agent's context window fastest?
- File reads and verbose tool results, followed by the definitions of every connected tool — those are present on every turn whether used or not. Trimming connected servers to the ones actually needed reclaims budget on every request.
Sources
Primary sources for the claims on this page. Specifications and vendor documentation change — verify against the source if a detail is load-bearing for a decision you are making.