Hand-Built Agent Memory and Context Compression in 200 Lines of TypeScript
Agent memory and context management are the difference between a demo that answers one question and a tool that can finish a multi-step programming task. This implementation shows how to separate complete history from working memory, use real API token counts to trigger compression, and generate structured summaries that preserve task state—all without framework lock-in.
A hand-built TypeScript agent, previously limited to single-turn interactions, gains two new capabilities without pulling in LangChain or any other framework. A `Session` class now holds the full message history within a single process, and a terminal readline loop lets users ask follow-up questions without restarting the program. The agent remembers previous turns because the same `Session` instance persists across prompts.
When the conversation grows too long for the model's context window, a `Compactor` module steps in. It uses a hybrid token-counting strategy—real API usage snapshots plus cheap UTF-8 byte estimates for new messages—to decide when to compress. The compactor calls the same LLM to produce a structured summary of older messages while keeping roughly the last 20,000 tokens of recent conversation in their original form. The summary preserves user goals, constraints, completed work, and next steps, so the model can continue a long task without losing critical context.
The result is a terminal agent that behaves like Claude Code or Codex CLI: start it once, ask a chain of related questions, and watch it automatically fold early history into a summary when the token budget is tight. The complete history remains intact in the `Session` object; only the working memory sent to the model is shortened.
The design cleanly separates 'what is saved' (complete history) from 'what is sent' (working memory), which makes future persistence or different compression strategies easier to add.
Using real API token counts as a baseline and only estimating new messages is a pragmatic middle ground—more accurate than pure estimation, less complex than integrating a tokenizer for every model.
Keeping recent messages in original form while summarizing older ones aligns with the 'lost in the middle' phenomenon observed in long-context LLMs, where models attend best to the beginning and end of the context.
The compactor's cut-point logic prioritizes conversation integrity over hitting an exact token target, which avoids breaking tool-call sequences that would cause API errors or confuse the model.