How Pi Manages Agent Context: Append-Only Session Trees, Projected Views, and Compaction Before the Window Breaks
Deep Dive Into Pi Source Code: How a Good Agent Manages Context?
theme: smartblue
In previous articles, we introduced Agent Loop and tool calling separately. This time, we continue digging into Pi's source code and discuss another core module of the Agent: the context management system.
What is a context management system?
It sounds abstract, but it actually solves very practical problems.
For example, how does the Agent manage conversation history? Every time an LLM is called, what exactly gets sent? When conversations get longer and tokens are about to exceed the Context Window limit, how should that be handled?
These are the problems the context management system needs to solve.
This article again uses Pi as an example, starting from source code, to see how a good Agent context management system is designed and implemented.
1. Overall Architecture
What core modules make up Pi's context management system?
Only 3 parts:
Message, Session, Context.
What are the differences and connections between Session, Message, and Context?
Let's briefly introduce these three things.
Message
First, Message.
Put simply,
Message has only four types:
export type Message = SystemMessage | UserMessage | AssistantMessage | ToolResultMessage;
The meanings are clear, so no need to explain one by one. It's fine if you don't understand here; this is just to give you a general idea.
Session
Session has quite a few more types:
export type SessionEntry =
| SessionMessageEntry
| ThinkingLevelChangeEntry
| ModelChangeEntry
| UsageEntry
| CompactionEntry
| BranchSummaryEntry
| CustomEntry
| CustomMessageEntry
| ContextEditEntry
| LabelEntry
| SessionInfoEntry;
In short,
Message can be wrapped as SessionMessageEntry and persisted to Session. But Session is not just a collection of Messages; it also contains other events.
For example, model switches, thinking level switches, and so on.
Context
Context has multiple levels of meaning,
But the overall hierarchy is roughly like this:
SessionProjection → AgentContext → TranscriptContext
You don't need to worry about what those mean yet.
Roughly, the relationship is:
Session (complete history)
→ select current branch, apply Compaction
→ Context (valid messages for this turn)
→ LLM
→ new Message
→ written back to Session
How do the context management system and the Agent Loop work together?
You can think of it this way:
The context management system is responsible for preparing "the materials the model can see this time," while the Agent Loop is responsible for taking those materials to call the model and execute tools.
The flow is roughly:
1. User sends message
2. Context management organizes the current valid history from Session and converts it into Context
3. Agent Loop sends Context to the LLM
4. LLM returns a response or Tool Call
5. Agent Loop executes the tool, gets the Tool Result
6. New messages or tool results are written back to Session
7. Context is rebuilt for the next round
On the code side, the flow is roughly:
SessionEntry[]
↓ branch selection, compaction, context edit
SessionProjection
↓
AgentMessage[]
↓ add executable tools
AgentContext
↓ transformContext
AgentMessage[]
↓ convertToLlm
Message[]
↓ normalizeContext
TranscriptContext
↓
Provider
But it's fine if you can't understand this yet. Being completely lost is normal, because I haven't explained anything yet — I've only described the composition of the context management system at a macro level.
Just remember: the context management system consists of 3 parts: Message, Session, Context.
Next, we'll introduce them one by one.
2. Message
Message is the basic data unit of the context management system. Session and Context both work around Message.
There are the following categories:
Standard Messages
These express messages the LLM can understand.
Their meanings are:
`SystemMessage`: carries the system prompt and its incremental changes, and can record changes in tool declarations.
`UserMessage`: user input.
`AssistantMessage`: model response, thinking content, Tool Calls.
`ToolResultMessage`: tool execution results.
Below, we focus on SystemMessage and UserMessage.
SystemMessage
Mainly used to record: System Prompt changes, tool additions or removals.
The process of generating SystemMessage has two cases: the first turn and subsequent turns.
First-turn generation chain:
AgentSession.prompt()
→ _startRun()
→ _prepareRun()
→ _preparePromptAndToolLoadout()
→ agent.prompt(messages)
→ runAgentLoop()
→ declareToolChanges()
→ message_end
→ prepareRequest()
→ first LLM request
Subsequent-turn generation chain:
runLoop()
→ prepareNextTurn()
→ AgentSession._installAgentNextTurnRefresh()
→ _preparePromptAndToolLoadout()
→ generate new SystemMessage when necessary
→ declareToolChanges()
→ emit message_end
→ persist
→ next LLM request
So the core is the _preparePromptAndToolLoadout() method and the declareToolChanges() method. The latter was actually introduced in the first article.
Regarding what these two methods do, the key is understanding these three things:
`agent.state.tools`: the tool code that can actually execute.
`SystemMessage.sections`: textual descriptions shown to the model.
`SystemMessage.toolsAdded/toolsRemoved`: the tool JSON Schema declarations given to the model.
_preparePromptAndToolLoadout() mainly handles the first two; the third is handled later by declareToolChanges().
Let's introduce them one by one.
First, the _preparePromptAndToolLoadout() method:
What does it do?
Assume the state before the call is:
Historical SystemMessage:
- Model knows tool: read
This turn's options:
- selectedTools: ["read", "bash", "unknown"]
Step 1: this._applyToolLoadout(options.selectedTools)
It looks up tools from the tool registry:
read → exists
bash → exists
unknown → doesn't exist, filtered out
Result after execution:
agent.state.tools = [
executable implementation of read,
executable implementation of bash
]
options.selectedTools = ["read", "bash"]
This step solves:
Which tools can the Agent Runtime actually execute?
Step 2: options.hiddenTools = [...this._hiddenDeclarations]
Purpose: determine which tools are visible to the model.
Normally:
options.hiddenTools = []
So the model sees:
["read", "bash"]
Suppose some extension hides bash:
options.selectedTools = ["read", "bash"]
options.hiddenTools = ["bash"]
Then the result is:
The model can only see: read
This step solves:
Among executable tools, which ones should be directly told to the model?
Because some tools cannot be directly revealed to the model; they need to be hidden.
Step 3: diffSystemPromptSections(previous, current)
First call getCurrentSystemMessage(messages)?.sections to calculate from historical Messages what the model currently knows:
Assume we get:
current sections:
{
tools: "read",
rules: "use read to read files",
cwd: "/old"
}
Then call buildSystemPromptSections(options) to build the target sections based on the new options:
Result is like:
target sections:
{
preamble: "You are an expert coding assistant...",
tools: "read、bash",
rules: "use read to read files; use bash to execute commands",
docs: "Pi documentation...",
cwd: "/new"
}
Then compare old and new content:
diffSystemPromptSections(previous, current)
Only the changed parts are kept:
diff:
{
tools: "read、bash",
rules: "use read to read files; use bash to execute commands",
cwd: "/new"
}
preamble and docs, if unchanged, are not re-sent.
This step solves:
Compared to what the model already knows, what Prompt content has changed this time?
Step 4: Generate an incremental SystemMessage
Final return:
{
role: "system",
content: "",
sections: {
tools: "read、bash",
rules: "use read to read files; use bash to execute commands",
cwd: "/new"
},
timestamp: Date.now()
}
Note: This is not the complete System Prompt; it's a patch.
Historical state plus patch:
old SystemMessage
+ new SystemMessage patch
= current complete System Prompt
If nothing changed, it returns undefined.
Now let's introduce declareToolChanges():
The sections.tools generated above is only the textual description shown to the model.
After entering the Agent Loop, declareToolChanges() will detect that bash was added, and supplement toolsAdded:
{
role: "system",
content: "",
sections: {
tools: "read、bash",
rules: "...",
cwd: "/new"
},
toolsAdded: [
{
name: "bash",
description: "...",
parameters: {/* JSON Schema */}
}
]
}
Complete call chain:
_preparePromptAndToolLoadout()
→ generate Prompt sections patch
→ declareToolChanges()
→ calculate actual tool changes
→ merge toolsAdded/toolsRemoved
→ get final SystemMessage
→ message_end
→ write to Session
→ enter first LLM request
So declareToolChanges()'s role is one, to supplement tool changes, and two, to supplement the tool execution schema.
Ok.
I've spent a lot of space on SystemMessage, because SystemMessage is important. Understanding its generation flow is very helpful for understanding the context management system.
And I also think this is a rather interesting mechanism.
UserMessage
Represents user input.
It actually has 2 sources:
- Normal user input
- Originally other AgentMessages that get converted to
UserMessagewhen sent to the LLM
First look at 1:
Its generation chain is:
AgentSession.prompt()
→ _startRun()
→ _prepareRun()
→ create UserMessage
→ agent.prompt()
Specifically it's created in _prepareRun():
Now look at 2:
As for AssistantMessage and ToolResultMessage, they're relatively simple, nothing much to say, so we'll skip them.
AgentMessage
It contains standard Messages plus extended special messages:
The extended messages mainly include these custom Messages:
`BashExecutionMessage`: represents a record of the user executing Bash via the `!` command, including command, output, exit code, and whether it was truncated.
`CustomMessage`: represents custom messages injected by extensions.
`BranchSummaryMessage`: represents a summary of the content of the branch being left when switching branches, so the model knows what happened in the abandoned branch.
`CompactionSummaryMessage`: represents the summary generated after old history undergoes Compaction, containing the summary text and the token count before compression.
These are ultimately converted into UserMessages.
Therefore:
AgentMessage
= standard Message
+ bashExecution
+ custom
+ branchSummary
+ compactionSummary
The conversion logic is in:
SessionEntry
This belongs to the Entry-layer concept.
Messages are wrapped as SessionEntries when persisted:
Summarizing:
AgentMessage
→ wrapped as SessionEntry, persisted to Session
→ SessionManager selects the current branch
→ get AgentMessage[]
→ convert to standard Message[]
→ compose the LLM Context
3. Session Management
Now let's introduce Session.
Core conclusion first:
Pi's Session is an append-only event tree stored in a JSONL file. Each record is linked via id + parentId, and the current leafId determines which branch is in use.
What is Session's core data structure?
Mainly SessionHeader and SessionEntry.
The first line of the JSONL is the SessionHeader, mainly recording file information about the Session:
export interface SessionHeader {
type: "session";
version?: number;
id: string;
timestamp: string;
cwd: string;
parentSession?: string;
}
Every line after the SessionHeader is a SessionEntry, recording specific Session node information.
export interface SessionEntryBase {
type: string;
id: string;
parentId: string | null;
timestamp: string;
}
These are the specific types:
Therefore,
Session is not just a Message list, but a complete conversation event log.
Session is managed by SessionManager:
The most critical parts are:
byId and leafId:
byId is used to quickly find parent nodes; leafId represents the current position.
JSONL file example
Simplified, it might look like this:
{"type":"session","version":3,"id":"session-1","cwd":"/project"}
{"type":"model_change","id":"a1","parentId":null,"provider":"anthropic","modelId":"model-a"}
{"type":"message","id":"a2","parentId":"a1","message":{"role":"system","content":"..."}}
{"type":"message","id":"a3","parentId":"a2","message":{"role":"user","content":"read package.json"}}
{"type":"message","id":"a4","parentId":"a3","message":{"role":"assistant","content":[]}}
{"type":"message","id":"a5","parentId":"a4","message":{"role":"toolResult","content":[]}}
Although the physical file is linear, parentId expresses a tree structure.
How are messages persisted and restored?
First, persistence.
After the Agent Loop produces a message, it emits:
the message_end event.
Using a finished tool call as an example:
Agent.runPromptMessages() passes processEvents() as the event receiver when calling the Agent Loop:
So processEvents() receives and handles the message_end event.
Upon receiving message_end, the Agent first updates its runtime state agent.state.messages:
At this point agent.state.messages already contains this message, but it may not have been written to the JSONL yet.
After processEvents() updates the state, it calls all subscribers in order:
Then it reaches the _handleAgentEvent method in AgentSession:
Continuing further:
The most important one is this _appendEntry method:
Inside this, the main one is the _persist method:
There's a detail here: writing to the file only happens when _hasConversation is true, meaning there is a UserMessage or AssistantMessage:
That completes persistence.
The full call chain is:
Agent Loop creates the final Message
→ emit(message_end)
Agent.processEvents()
→ agent.state.messages.push(message)
→ call subscribers
AgentSession._handleAgentEvent()
SessionManager.appendMessage()
→ create SessionMessageEntry
SessionManager._appendEntry()
SessionManager._persist()
Session uses append-only mode
For example, to hide an old message, Pi doesn't directly modify the old record in the JSONL. Instead, it appends:
ContextEditEntry {
targetId: "old message ID",
replacement: null
}
To change the Session name, it also appends a new session_info.
This way, history is not damaged, and it's easy to audit and restore.
How is the Session Tree and branching mechanism implemented?
How the Tree forms
Every time an Entry is appended:
new Entry.parentId = current leafId
current leafId = new Entry.id
How to get the current branch
Mainly this method:
For example, the complete events are:
A → B → C → D → E
Tracing back from E:
E → D → C → B → A
Finally reversed:
A → B → C → D → E
This is the current valid branch.
How to create a branch
Suppose the original tree is:
A → B → C → D → E
The user wants to go back to C and continue from there.
Call:
branch("C") does only one thing:
leafId: E → C
It doesn't delete D or E, and it doesn't immediately write a new Entry.
The next time a message F is appended:
F.parentId = C
leafId = F
Resulting in:
About branchWithSummary
For example, the complete events are:
A → B → C → D → E
Now we want to cut a new branch starting at C.
Normal branch() directly switches to old node C;
D and E don't enter the new Context.
If you want the model to know what happened on the abandoned branch, you can use:
branchWithSummary()
For example:
Old branch D → E:
tried to modify auth code, but tests failed
After switching back to C, add:
BranchSummaryEntry:
"Previously attempted to modify auth code, but tests failed. Reverted."
The tree becomes:
A → B → C → D → E
\
S(branch_summary) → F
The new Context is:
A → B → C → BranchSummary → F
This doesn't keep all the details of the old branch, but preserves important conclusions.
The difference between getTree() and buildSessionContext()
getTree():
Returns the complete Session Tree, including all branches.
Primarily used for:
Session Tree UI, branch selection, debugging and browsing history.
buildSessionContext():
Only returns the valid branch corresponding to the current leaf.
Primarily used for:
Restoring the Agent, building LLM Context, Compaction, next model request.
4. Context Construction
How is context built each time the LLM is called?
The core is one sentence:
Before every LLM call, Pi rebuilds Context from the current Session branch; tool calls and results are written back to the Session as normal messages and participate in the next round of reasoning.
Call chain:
Session Tree
→ buildSessionProjection()
→ current branch's AgentMessage[]
→ prepareRequest() replaces AgentContext.messages
→ transformContext()
→ convertToLlm()
→ normalizeContext()
→ LLM
Let's introduce them one by one:
First: buildSessionProjection()
It mainly does five things:
1. Get the current branch
const path = buildSessionPath(entries, leafId, byId);
Trace from the current leafId back to the root along parentId:
complete tree: A → B → C → D
↘ E → F
leafId = F
current path = A → B → C → E → F
2. Restore model and Thinking Level
const { thinkingLevel, model } = getSessionContextSettings(path);
Walk through the model_change, thinking_level_change, and the model recorded in AssistantMessages on the current branch to get the current configuration.
3. Apply Compaction
const contextEntries = buildContextEntries(entries, leafId, byId);
If there's no Compaction:
contextEntries = current complete branch
4. Apply Context Edit
First collect the context_edit entries on the current branch:
const edits = new Map<string, ContextEditEntry>();
Then process each Entry:
replacement = null
→ that message is removed from Context
replacement = { content: "new content" }
→ replace the message content
no edit
→ keep the original message
When multiple edits point to the same message, the later edit takes effect.
This seems to come out of nowhere — where did this edit suddenly come from?
What is Context Edit
Its purpose is: finding which old messages need modification
Suppose the Session's original records are:
E1: UserMessage("delete database")
E2: AssistantMessage("preparing to execute")
E3: ContextEdit(targetId=E1, replacement="read database")
Pi doesn't modify E1, because Session is append-only. Instead, it appends E3.
buildSessionProjection() first establishes:
edits = {
E1 → E3
}
When processing E1, it finds an Edit exists, so the projection result becomes:
original Session:
UserMessage("delete database")
final Context:
UserMessage("read database")
The original E1 in the JSONL is still "delete database"; only the model sees the replaced version.
If:
ContextEdit(targetId=E1, replacement=null)
Then the projection result for E1 is:
[]
Meaning it doesn't enter the Context.
The ContextEdit itself is also not sent to the model as a message.
5. Convert and merge Messages
Each SessionEntry goes through:
projectContextEntry()
→ sessionEntryToContextMessages()
converted into zero, one, or multiple AgentMessages:
Why convert Entries to Messages
Because not all SessionEntries are messages.
For example:
ModelChangeEntry
LabelEntry
ThinkingLevelChangeEntry
SessionInfoEntry
These are Session state; the LLM shouldn't see them.
How prepareRequest executes
Its role is:
Before each actual LLM call, replace the messages the Agent Loop currently holds with the latest messages computed by the SessionManager.
Where it executes:
Then it gets here:
Thus rebuilding the projection.
Why replacement is needed
The Agent Loop starts with a snapshot:
AgentContext.messages = messages at Agent Loop startup
But during execution, things can change:
branch or summary changes, Compaction, Context Edit, new messages already persisted to Session, etc.
So before the LLM request, the latest projection from Session should be authoritative.
Example
The Agent Loop's original Context:
currentContext.messages = [
UserMessage("old question"),
AssistantMessage("old answer")
]
Meanwhile, Compaction happened in the Session:
SessionProjection.messages = [
CompactionSummaryMessage("previously discussed auth design"),
UserMessage("continue implementing")
]
After prepareRequest():
currentContext.messages = [
CompactionSummaryMessage("previously discussed auth design"),
UserMessage("continue implementing")
]
Next, streamAssistantResponse() calls the LLM with the replaced Context.
Full flow:
Agent Loop prepares request
→ prepareRequest()
→ buildSessionProjection()
→ get latest messages
→ replace currentContext.messages
→ convertToLlm()
→ call LLM
transformContext()
It can modify Messages once more before the request is sent to the LLM.
Example:
Suppose the messages built from Session are:
[
SystemMessage("You are a coding assistant"),
UserMessage("read the password file"),
AssistantMessage("ok"),
UserMessage("continue")
]
Before sending to the LLM, transformContext() could temporarily turn them into:
[
SystemMessage("You are a coding assistant; do not read sensitive files"),
UserMessage("continue")
]
The difference between the three:
prepareRequest()
handles: getting the correct historical messages from Session
transformContext()
handles: temporarily deciding what the model sees in this request
convertToLlm()
handles: converting AgentMessages into Messages the LLM supports
It's essentially a hook layer.
convertToLlm()
Its role:
Convert Pi's internal AgentMessage[] into the standard Message[] the LLM can understand.
Why conversion is needed
Pi internally has these extended messages:
bashExecution
custom
branchSummary
compactionSummary
But the LLM layer only recognizes:
system
user
assistant
toolResult
So conversion is needed.
The specific conversion logic was already covered earlier, so I won't repeat it here.
normalizeContext()
convertToLlm() has already prepared the message content:
[
{ role: "user", content: "hello" }
]
But the Provider doesn't receive a bare message array; it receives a unified request object:
{
messages: [
{ role: "user", content: "hello" }
]
}
normalizeContext() handles this wrapping.
It's the final step before the LLM request.
Which messages enter the Context Window?
Mainly these filters:
Full determination chain:
buildSessionPath()
selects the current branch
→ buildContextEntries()
applies Compaction
→ buildSessionProjection()
applies Context Edit, Entry-to-Message conversion
→ transformContext()
extension temporary modifications
→ convertToLlm()
filter or convert special messages
→ final Context Window
In short, remember this sentence:
The Context Window does not receive the complete Session; it receives the valid projection of the current branch.
5. Compaction (key section)
Compaction is basically compression.
It doesn't delete Session history; it appends a CompactionEntry, replacing older messages with a summary while keeping recent messages verbatim.
Why is Compaction needed? When does it trigger?
The model has a fixed contextWindow. As history grows, requests may exceed the model's limits.
So Compaction is needed.
Trigger conditions:
Timing of triggers:
Compaction has four main trigger points:
1. After the user submits a new prompt, before requesting the LLM
User calls prompt()
→ _prepareRun()
→ _checkCompaction()
→ summarize first if necessary
→ then create and send the new UserMessage
At this point the user has already submitted input, but the new UserMessage hasn't been added to the Agent Loop yet. This mainly handles a Context that's already too long after a previous interruption.
2. After the Assistant response completes
AssistantMessage completes
→ _handlePostAgentRun()
→ _checkCompaction()
→ summarize immediately if necessary
This is the most common automatic trigger point, because the AssistantMessage contains the latest token usage, which allows accurately determining whether Context is close to the limit.
3. After a Tool Call completes, before preparing the next LLM request
Assistant Tool Call
→ ToolResult
→ prepareNextTurn()
→ _compactBeforeNextAssistantResponse()
→ summarize if necessary
→ next LLM request
Prevents oversized tool results from causing the next request to exceed the Context Window.
4. User manually triggers
/compact
→ AgentSession.compact()
No need to wait for a new user message or model response.
Those are the four trigger points.
Specific implementation flow
First, it still works based on the current branch, so the first step is buildSessionProjection:
The second step is to find the split point from back to front:
Pi accumulates tokens from the newest message backward, aiming to keep approximately keepRecentTokens.
A detail here: the split point won't be placed on a ToolResult, because a ToolResult must follow its corresponding Tool Call.
Once the split point is selected, content before the split point is summarized; content after is kept.
The summary is implemented by compact(), which is essentially one LLM call. Then appendCompaction() generates the CompactionEntry and appends it to the Session Tree.
Summarizing:
`shouldCompact()` decides whether to compress;
`prepareCompaction()` decides which messages to compress;
`compact()` calls the LLM to generate the summary;
`appendCompaction()` saves the summary and makes it available to the subsequent Agent Loop.
How does the compressed context participate in the subsequent Agent Loop?
After compression completes, the Agent Loop doesn't need to understand Compaction. It only sees a new set of AgentMessage[].
First, after executing the compression, _refreshFinalizedContext() is called once:
Therefore:
agent.state.messages
is replaced with the compressed messages.
A concrete example:
Before compression:
agent.state.messages = [
System,
User1,
Assistant1,
User2,
Assistant2,
User3,
Assistant3
]
After compression:
User1 through Assistant2 → summary: "The user is implementing login functionality; database design has been completed."
Recent messages User3, Assistant3 are kept.
Then _refreshFinalizedContext() replaces the Agent's history:
agent.state.messages = [
SystemCheckpoint,
CompactionSummary(
"The user is implementing login functionality; database design has been completed."
),
User3,
Assistant3
]
The key point: the old User1 through Assistant2 no longer appear in agent.state.messages.
They remain in the Session file, but no longer enter the Agent Loop.
Then the user continues sending messages.
For example, the user enters:
continue implementing the login API
The Agent Loop gets:
[
SystemCheckpoint,
CompactionSummary("The user is implementing login functionality; database design has been completed."),
User3,
Assistant3,
UserMessage("continue implementing the login API")
]
The Agent Loop doesn't know Compaction happened here. To it, CompactionSummaryMessage is just a normal historical message.
Then it's sent to the LLM:
convertToLlm() converts CompactionSummary into a UserMessage:
[
SystemMessage,
UserMessage(
"Previous conversation has been compressed. Summary:
The user is implementing login functionality; database design has been completed."
),
User3,
Assistant3,
UserMessage("continue implementing the login API")
]
So the complete process is very simple:
Before:
all historical messages → Agent Loop
After compression:
historical summary + recent verbatim → Agent Loop
Therefore, Compaction doesn't directly control the Agent Loop; it just replaces the messages used by the Agent Loop in advance.
The Agent Loop's subsequent model calls and Tool Calling flow are completely unchanged.
Complete source code call chain
shouldCompact() = true
→ AgentSession._runAutoCompaction()
→ prepareCompaction()
→ AgentSession._runDefaultCompaction()
→ compact()
→ generateSummaryWithUsage()
→ completeSummarization()
→ call LLM
→ appendCompaction()
→ _refreshFinalizedContext()
6. Architecture Summary
Pi's context management system,
I think it can be divided into four layers: persistence layer, projection layer, execution layer, Provider layer.
I used AI to draw a diagram, take a look:
Pi's context management system complete running flow
When Pi starts, it creates or reads a JSONL Session, rebuilds the current branch based on parentId, and restores Agent.state.messages via buildSessionProjection();
After the user submits a Prompt, AgentSession starts the Agent Loop. All final messages, at message_end, first update Agent memory, then persist as SessionEntries;
Before every LLM call, prepareRequest() re-projects the Session, replacing AgentContext.messages with the latest result, then sequentially goes through transformContext(), convertToLlm(), and normalizeContext(), before the Provider calls the specific model;
If the model returns a Tool Call, the Agent Loop executes the tool, generates and persists the ToolResult, then enters the next round of requests;
If Context approaches the limit or an Overflow occurs, Pi calls a summarization LLM, compresses older messages into a Compaction Summary, keeps the System checkpoint and recent verbatim content, refreshes Agent.state.messages, then continues running or waits for the next input.
This forms the complete loop of:
Session persists facts → Context projects → Agent Loop executes → new messages written back to Session
Core design ideas
Complete history is separated from model context
Session = complete facts
Context = the projection shown to the model this time
This supports both a complete, recoverable, branchable fact log and the limited Context Window.
Append-only
History isn't modified directly; it's extended via appends, ensuring history is auditable and recoverable.
What are the pros and cons of this design, and what's worth borrowing?
Pros:
Strong recoverability
Good protection of the Context Window Size
Strong extensibility
Cons:
Many layers; high understanding cost
JSONL grows continuously
Summarization itself has a cost, and is lossy and non-deterministic
Most worth borrowing:
Session is responsible for saving complete history; Context is responsible for deciding what the model sees.
Decoupling the two is the design idea most worth borrowing from Pi's context management system.
Afterword
This is my third article studying Pi's source code. The first two introduced Agent Loop and tool calling.
After this research, I increasingly feel that:
The core of Agent development isn't just calling a large model, but building a reliable operating mechanism around the large model.
Compared to just learning frameworks and APIs, I think understanding these underlying design ideas has more long-term value.
I'll continue studying Pi's other core modules in the future and share my understanding.
Top 1 from juejin.cn, machine-translated. The original thread is authoritative.
Context management is the hidden ceiling of Agents, Pi's layered compression approach is worth a closer look. In practice, the thing that most affects results is the truncation strategy for tool results, which is more sensitive than switching models. I often find this kind of deep source-code analysis on AI345, and I also collect related tools and tutorials together, so I don't have to search when I use them.