Memory & Stateintermediate12 min · 9 steps
Memory Architectures
Split an agent's memory into working, episodic and semantic tiers, decide when memories move between them, and prove each layer beats plain retrieval.
tokendatarequest
ProblemStep 1 of 9
Why not keep everything in a 1M-token window?
Frontier models now read about 1M tokens, so it is tempting to paste every past chat, tool output and document into each call. It works, but it is wasteful on three counts.
Cost: a 1M-token call to Claude Opus 5.5 is $4.00 of input uncached and $0.20 even fully cached, every turn. A 10K-token call with retrieved context is $0.04. Attention: clutter degrades reasoning (lost in the middle, context rot). Latency: time to first token grows with uncached context.
Watch outCaching narrowed the gap but did not close it. Keep the window lean and retrieve what the current task needs.
← Previous lessonReal-World Agents: AutoGPT to Agentic RAGNext lesson →Short-Term Context Management
Lesson 1 of 6 in Memory & State