How Pi Manages Agent Context: Append-Only Session Trees, Projected Views, and Compaction Before the Window Breaks
Coding agents die when context overflows, and Pi's design is a working answer to that problem: separate what happened from what the model sees. The append-only session plus projection pattern makes branching, summarization, and recovery practical without destroying history, and the diff-based system prompt shows a concrete way to stop leaking tokens on long sessions.
Pi's context management breaks down into three pieces: Message, Session, and Context. The Session is an append-only JSONL event tree where each record links via id and parentId, and a leafId marks the current branch. Before every LLM call, buildSessionProjection() walks that branch and produces the message list the model will see, applying branch selection, compaction, and context edits along the way. The Agent Loop never holds the whole history; it gets a snapshot that prepareRequest() replaces with the latest projection right before each call.
The system prompt is generated as a patch rather than a full prompt each turn. _preparePromptAndToolLoadout() diffs old and new prompt sections so unchanged parts like the preamble and docs are not resent, and declareToolChanges() appends only the tool JSON schemas that actually changed. Session branching works by changing leafId to an earlier node, and branchWithSummary() injects a summary of the abandoned branch so the model knows what it missed. ContextEdit entries can replace or remove old messages in the projection without ever touching the original JSONL record.
Compaction triggers at four points: after a user prompt before the first LLM request, after an assistant response completes, before the next request following a tool call, and manually via /compact. It doesn't delete history; it appends a CompactionEntry containing an LLM-generated summary of older messages, keeps recent messages verbatim, avoids splitting on ToolResult, and refreshes agent.state.messages so the Agent Loop never needs to know compaction happened.
Diff-based SystemMessage generation is a production-grade token optimization; resending an unchanged preamble on every turn would quietly eat the context window on long sessions.
The architecture makes a deliberate trade: append-only history is auditable and recoverable, but the JSONL grows without bound and every compaction is a lossy, non-deterministic LLM call.
Keeping the Agent Loop ignorant of compaction is a clean boundary; the loop just sees a new message array, which keeps the execution layer simple at the cost of a heavier context layer.
ContextEdit is a clever append-only trick: the model sees a corrected version of an old message, but the file still records what was actually sent, which keeps the history honest for debugging.
Context management is the hidden ceiling of Agents, Pi's layered compression approach is worth a closer look. In practice, the thing that most affects results is the truncation strategy for tool results, which is more sensitive than switching models. I often find this kind of deep source-code analysis on AI345, and I also collect related tools and tutorials together, so I don't have to search when I use them.