Memory & Stateintermediate11 min · 8 steps
Short-Term Context Management
Keep the context window lean, cacheable and append-only: paged KV memory, prefix caching, sliding windows, summaries and selective dropping.
tokendatarequest
ProblemStep 1 of 8
Two windows, and the smaller one fills first
The model context window is the hard architectural limit (1.05M tokens on GPT-6.1 Sol, for example). The application context window is a budget you set, such as 64K, because attention cost grows with context and quality degrades well before the hard limit.
Every turn adds history and tool output. Soon the next message no longer fits, and you still need a buffer zone for the reply. On reasoning models that buffer must include thinking tokens, which count against max_tokens.
Watch outPlan the budget before you hit it: decide what gets dropped, summarised or offloaded, and in what order.
Lesson 2 of 6 in Memory & State