跪拜 Guibai
← All articles
Artificial Intelligence · Full Stack · AI Programming

How Pi Manages Agent Context: Append-Only Session Trees, Projected Views, and Compaction Before the Window Breaks

By Rater小宇宙 ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Coding agents die when context overflows, and Pi's design is a working answer to that problem: separate what happened from what the model sees. The append-only session plus projection pattern makes branching, summarization, and recovery practical without destroying history, and the diff-based system prompt shows a concrete way to stop leaking tokens on long sessions.

Summary

Pi's context management breaks down into three pieces: Message, Session, and Context. The Session is an append-only JSONL event tree where each record links via id and parentId, and a leafId marks the current branch. Before every LLM call, buildSessionProjection() walks that branch and produces the message list the model will see, applying branch selection, compaction, and context edits along the way. The Agent Loop never holds the whole history; it gets a snapshot that prepareRequest() replaces with the latest projection right before each call.

The system prompt is generated as a patch rather than a full prompt each turn. _preparePromptAndToolLoadout() diffs old and new prompt sections so unchanged parts like the preamble and docs are not resent, and declareToolChanges() appends only the tool JSON schemas that actually changed. Session branching works by changing leafId to an earlier node, and branchWithSummary() injects a summary of the abandoned branch so the model knows what it missed. ContextEdit entries can replace or remove old messages in the projection without ever touching the original JSONL record.

Compaction triggers at four points: after a user prompt before the first LLM request, after an assistant response completes, before the next request following a tool call, and manually via /compact. It doesn't delete history; it appends a CompactionEntry containing an LLM-generated summary of older messages, keeps recent messages verbatim, avoids splitting on ToolResult, and refreshes agent.state.messages so the Agent Loop never needs to know compaction happened.

Takeaways
— The Session is an append-only JSONL event tree; each entry carries id and parentId, and leafId determines which branch is active.
— Message has only four standard types: SystemMessage, UserMessage, AssistantMessage, and ToolResultMessage.
— SystemMessage generation diffs old and new prompt sections; unchanged sections like preamble and docs are not resent.
— The system prompt sent to the model is a patch: old SystemMessage plus new patch equals the current full prompt.
— declareToolChanges() separately appends toolsAdded/toolsRemoved with JSON Schemas, distinct from the textual sections.
— prepareRequest() re-projects the Session before every LLM call, replacing the Agent Loop's snapshot with the latest projection.
— Compaction triggers at four points: after user prompt submission, after assistant response completion, before the next LLM request after a tool call, or manually via /compact.
— Compaction appends a CompactionEntry instead of deleting history; older messages are replaced by a summary while recent messages stay verbatim.
— The compaction split point never lands on a ToolResult, because a ToolResult must follow its corresponding Tool Call.
— ContextEdit entries can replace or null out an old message in the projection without modifying the original JSONL record.
— The pipeline is buildSessionProjection() → transformContext() → convertToLlm() → normalizeContext() → Provider.
— The Context Window receives the projection of the current branch, not the complete Session.
Conclusions

Diff-based SystemMessage generation is a production-grade token optimization; resending an unchanged preamble on every turn would quietly eat the context window on long sessions.

The architecture makes a deliberate trade: append-only history is auditable and recoverable, but the JSONL grows without bound and every compaction is a lossy, non-deterministic LLM call.

Keeping the Agent Loop ignorant of compaction is a clean boundary; the loop just sees a new message array, which keeps the execution layer simple at the cost of a heavier context layer.

ContextEdit is a clever append-only trick: the model sees a corrected version of an old message, but the file still records what was actually sent, which keeps the history honest for debugging.

Concepts & terms
Compaction
A process that calls a summarization LLM to compress older conversation messages into a summary, appends a CompactionEntry to the Session, and keeps recent messages verbatim so the agent stays under the context window.
SessionProjection
The result of transforming the full Session event tree into the concrete message list the model will see for this call, applying branch selection, compaction, and context edits.
ContextEdit
An append-only mechanism that changes what the model sees for an older message: a replacement swaps the content in the projection, and a null replacement removes the message entirely, without modifying the original JSONL record.
leafId
The id of the current position in the Session tree; changing it to an earlier node is how Pi creates a new branch without deleting anything.
From the discussion
Featured comments
咬代码的兽

Context management is the hidden ceiling of Agents, Pi's layered compression approach is worth a closer look. In practice, the thing that most affects results is the truncation strategy for tool results, which is more sensitive than switching models. I often find this kind of deep source-code analysis on AI345, and I also collect related tools and tutorials together, so I don't have to search when I use them.

See top comments, translated →
Source: juejin.cn ↗ Google Translate ↗ Backup ↗