How Pi's Harness Keeps Agent Sessions Alive Across Crashes
Most agent frameworks treat session persistence as an afterthought — dump the message array to JSON. Pi's design shows what's actually required: a phase-aware write strategy, a pending queue for mid-turn config changes, and append-only compaction that never mutates history. The architecture is small (five phases, one queue, one tree) but covers session recovery, model switching, context compaction, and crash handling under a single set of mechanisms.
Pi's agent harness solves a problem most agent frameworks dodge: how to persist a session when half the runtime — tool functions, provider instances, hook handlers — can't be serialized. The answer is a semi-durable harness that draws a hard line: data goes to the session log, code stays with the host. A five-phase state machine (idle, turn, compaction, branch_summary, retry) governs when writes can happen, and a pending queue ensures config changes made mid-turn don't corrupt the write order.
Context compaction is implemented as another log append, not a destructive edit. A compaction entry stores a summary and a `firstKeptEntryId` boundary; during projection, messages before that boundary are replaced by the summary while the raw log remains intact. Incremental summarization keeps compaction cost bounded even as sessions grow indefinitely.
Recovery rebuilds context by walking the append-only tree from leaf to root, applying the latest compaction boundary, and projecting the result into a message array. Interrupted turns follow the normal finalization path — aborted messages are written to disk with an aborted status, so nothing is lost.
Pi's explicit admission that full persistence is impossible — and its decision to design around that constraint rather than chase it — is more valuable than any technical trick in the implementation. Most frameworks quietly ignore the serialization problem.
The pending queue for mid-turn config changes solves a subtle ordering bug that most ad-hoc session implementations would miss: if a config entry lands before the messages it should follow, recovery replay breaks.
Compaction as an append rather than a mutation means the session log is an immutable audit trail. You can always reconstruct what the model actually saw at any point, which matters for debugging and evaluation.
The five-phase state machine governs session consistency, not UI progress. The fact that a UI can display the phase is explicitly called out as a side effect — a discipline most frameworks lack.
Thariq Shihipar's 'capability overhang' concept — that harnesses designed for weaker models actively suppress stronger ones — suggests that as models improve, the harness should shrink, not grow.