Context Windows Are RAM, Not Disk: The Engineering Playbook Behind Production Agents
Agent runs that hit hundreds of tool calls collapse without deliberate context management, and naive fixes such as a bigger window or hard-truncating history quietly break results: forgotten constraints, poisoned facts, hallucinated file paths. These techniques are the difference between a demo that works and an agent that survives a real codebase.
Context rot is the name for what happens when you stuff a model's window: the more tokens, the worse recall of intermediate facts. Anthropic says to treat context as a finite resource, and mainstream agents now engineer each step to feed only the minimal high-signal token set. That means four strategies: Write information to external storage (scratchpads, CLAUDE.md-style rule files, long-term memory services), Select only relevant files and tools each turn, Compress when the window approaches 80-95% (tool result clearing, auto-compact summaries, hierarchical summarization), and Isolate contexts by splitting work into sub-agents, sandboxes, and state schemas.
Product implementations show the same trade-offs. Claude Code preloads CLAUDE.md, relies on on-demand Read/Glob/Grep, auto-compacts at 95%, and can spawn sub-agents. Cline deliberately avoids codebase indexing because RAG indices go stale; it uses a Memory Bank of markdown and progressive disclosure of large results. Aider builds a tree-sitter-based Repo Map with importance scoring. LangGraph uses checkpointer state and summarization nodes. OpenAI handles threads server-side with automatic truncation.
Two costs often get overlooked. Tool results are the biggest source of bloat; the principle is to return indexes and samples, not full data, and let the model call the tool again for more. Prefix caching is the cheapest lever: keep system prompt, tools, and memory files fixed and in order, and input token cost drops to 10-20% of original, saving 5-10x on long-session bills. A pre-launch checklist covers fixed prefix share under 30% of the window, explicit truncation strategies, automatic compression before 80%, isolated cross-session memory, and regression testing on an offline case library.
Scaling context windows to 200k or 1M tokens does not remove the need for context management, because context rot degrades recall as tokens accumulate; the source states this as a core judgment, so the practical takeaway is that window size optimization has a ceiling.
The biggest cost lever in multi-turn agents is often input tokens, not output. With prefix caching, long sessions can be 5-10x cheaper; this reframes contextual engineering as a billing and latency issue, not just a correctness one.
Cline's explicit refusal to index codebases because RAG indices go stale suggests a fundamental tension: precomputed indexes optimize first-turn latency but decay with code changes, while on-demand navigation trades latency for freshness.
Compression introduces a correctness risk: summaries can drop details that later prove critical, so the guidance to tune recall before precision is a practical acknowledgment that context management is a quality trade-off, not a free win.