跪拜 Guibai
← All articles
Artificial Intelligence

120 Runs, Five Session Strategies: No Universal Winner in Agent Handoff Experiment

By FridaySunny ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Choosing whether to continue, compact, fork, or start fresh with a summary can change agent run costs and latency by more than 10x on the same task, without any guaranteed delivery benefit. The results invalidate the simple "more context is better" assumption and push teams to measure handoff strategy per model and per workload instead of trusting a default.

Summary

FridaySunny's experiment compared five handoff strategies — fresh session, continue, compact, fork, and fresh-plus-short-summary — across two models, two old task histories, and two target tasks, with three repeats each. The headline result: no strategy wins everywhere. In the Sol complex-history E2E group, continue sessions cost $2.29 on average and took 14 minutes versus $0.19 and 2.7 minutes for a fresh session, yet all three delivery outcomes were identical (3/3). On DeepSeek's complex E2E task, the pattern flipped: fresh took 6.8 minutes while continue took 3.0 and fork 2.2, all completing fully.

Cost analysis shows why full history can be expensive even when cache hit rates are high. In the Sol E2E group, continue sessions averaged $1.266 in cache-read fees alone, versus $0.194 total for a fresh task, because roughly 6.33 million cached tokens were re-read across many calls. Long-input pricing surcharges accounted for about 47–49% of recorded costs for continue and fork in that group. Short summaries matched fresh on Sol's E2E and beat it on DeepSeek's E2E, but on bugfix tasks they failed functional acceptance 2 out of 6 runs versus 0 for fresh.

The experiment also separates delivery failures: 102 of 120 runs fully qualified; 14 failed by runtime error or timeout, and 4 had functional boundary defects. The author is explicit about limits — three runs per group, one fixed history per round, uncontrolled cache warmth, and API-equivalent pricing — and concludes that session strategy must be evaluated per model, per task, and per delivery requirement, not by context length alone.

Takeaways
— 120 runs compared new session, continue, compact, fork, and summary+fresh across two models, two target tasks, and three repeats.
— On Sol with a complex history and E2E task, continue and fork cost 11–19x more and took about 5x longer than a fresh session, with identical 3/3 delivery.
— DeepSeek's complex E2E reversed the cost picture: fresh took 6.8 minutes while continue took 3.0 and fork 2.2, all passing fully.
— Cache-read costs alone for continue averaged $1.266 per run versus $0.194 total for a fresh session, a 6.5x task-level ratio.
— Continue sessions read about 6.33 million cached tokens per run, because old history re-entered input across many calls.
— Long-input pricing surcharges accounted for roughly 47–49% of recorded costs for continue and fork in the Sol complex E2E group.
— Summary+fresh matched fresh on Sol's E2E and was cheaper/faster on DeepSeek's E2E, but missed functional acceptance on bugfix tasks (4/6 vs 6/6 for fresh).
— 102 of 120 runs fully passed; 14 had runtime/timeout failures and 4 had functional boundary defects.
— The experiment used one fixed development history per round with only three repeats, so it cannot establish context-length thresholds.
— Costs are API-equivalent estimates at frozen rates, not current prices or actual subscription bills.
Conclusions

The expensive part of full-history handoff is not a single large input but the cumulative re-sending of old context across many tool calls, so cache hits mask a real cost problem.

Model behavior flips the cost equation: DeepSeek gets faster with inherited history while Sol gets slower, meaning handoff strategy is a per-model tuning knob, not a universal rule.

Attributing cost growth to "context gravity" is wrong when long-input pricing surcharges alone inflate continue and fork costs by nearly half; that is a pricing artifact, not model degradation.

Short summaries lose on bugfix tasks because the desensitization rules exist only in the old chat; delivery success cannot be inferred from cost and time improvements alone.

Separating runtime failures from functional defects matters: calling all 18 incomplete runs "code failures" would disguise that 14 were operational issues, not model reasoning problems.

Concepts & terms
Compact
A session handoff strategy that uses the model provider's native conversation compression before continuing, aiming to reduce context length while preserving key information. In the experiment it cut costs versus full continue but still lost to fresh on delivery in one group.
Fork
A session handoff strategy that copies the entire old session history into a new session identity. It retains full context but can multiply cost and time when that history is replayed repeatedly across calls.
Context gravity
The hypothesis that longer conversation history inevitably pulls down speed and raises cost. The experiment found no consistent threshold for this effect; part of the observed cost increase came from pricing tiers, not model behavior.
From the discussion
Featured comments
执着的勺子

最好一个模块任务做完后新开对话,然后把任务状态和进度写进文档中,这样既不会丢失进度,也不会遇到上下文爆炸的问题。试过一直用工具自带的压缩,没有用,大模型思考2轮就得压缩一次,太浪费时间。

FridaySunny

按模块完成后交接,我认为关键是把后续任务真正需要的信息留下,而不是把旧聊天完整搬过去。你提到反复压缩的等待,也应该算进任务总耗时。 我们这轮“新开”组仍能读相同的代码、文档和任务说明,所以比较的是要不要额外继承聊天历史。同一轮 E2E 中,Sol 新开更省时,DeepSeek 却是继承历史更快;短摘要也没有稳定胜过直接新开。每组只有三次,暂时还不能得出所有任务都该新开的结论。 如果把交接落到文档里,我会建议除了状态、进度,再写清已确认的约束、关键决策及原因、验证过什么、还有哪些未完成;尤其区分“试过失败”和“只是讨论过”。下一轮也可以专门检查这些决策有没有保留下来,而不只看摘要是否流畅。 你现在写回文档的是进度清单,还是也会保留被否决的方案和原因?这部分可能更能解释新会话能否顺利接手。

执着的勺子  → FridaySunny

我用的是spec-superflow这个插件,插件会自动设置任务状态和进度,新开聊天会查询上次任务的节点,然后接着执行。

See top comments, translated →
Source: juejin.cn ↗ Google Translate ↗ Backup ↗