跪拜 Guibai
← All articles
Frontend · Artificial Intelligence · Agent

Agents That Never Error Are the Scariest: 10 runtime pitfalls from 6 months of building a coding agent

By 樊小肆 ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Anyone building agents that run unattended will hit these same silent failure modes: the model isn't the weak link, the plumbing around it is. The fixes here translate directly into cheaper, more reliable agent pipelines for Western teams, especially around tool result validation and context-window management.

Summary

Building a Claude Code-style terminal agent on DeepSeek, the author logged 129 bug-fix commits out of 359 total over six months. The recurring lesson: when an agent goes wrong, it usually doesn't throw an error. It streams output, calls tools, and says "done" while quietly producing wrong results.

The article catalogs 10 distinct failure modes, each with a deceptively normal surface. A tool that "spins forever" was actually a missing end-of-execution signal. A cleared input box made the user believe a message sent, but the backend silently discarded it during generation. A model declaring "basically done" after half the work turned out to be early wrap-up from a too-loose ending rule. A command that failed still looked successful because the tool output lacked an exit code, and the model guessed success from output text alone.

Other pitfalls include a search tool used like a dictionary, wasting 20% of rounds and tripling token use; a model copying a malformed site: query because the guidance lacked an example; a directory tool silently returning the wrong tree, producing a three-layer lie chain; an approval system spamming 100+ confirmations due to a name mismatch; long tasks losing constraints during context compression; and a cache block that invalidated whenever any byte changed, dropping hit rate to 15%. Each fix is concrete: add sentinels and fallbacks, queue instead of reject, measure character counts instead of guessing semantics, provide copyable templates, unify names, split entity indices from narrative in summaries, and move variable content out of pinned cache blocks.

Takeaways
— Agent failures usually present as normal behavior, not errors: streamed output, tool calls, and polite "done" messages can all hide real breakage.
— Missing end-of-execution signals cause tools to spin indefinitely; fixes include catch-all exception paths, session-recovery reconciliation, and state reset in finally blocks.
— The "clear the UI first, then call" pattern can silently discard user input during generation; queue input instead of rejecting it, and surface a message when queuing fails.
— When classifying whether a model is done, use measurable thresholds like reply length or round counts, not keyword regexes on semantic words like "done".
— Command results must carry exit codes as machine-judged success signals; never let the model infer success from output text.
— Behavioral statistics revealed 20% of agent rounds were wasted on repeated reads and searches; nudges in context, plus prompt rules, cut redundancy dramatically.
— Guidance for models is most effective when it includes a copyable template (e.g., `site:${domain} keyword`) rather than just a syntax name.
— Tools must not silently fall back to alternate data when a parameter is missing; explicit failure lets the model self-correct, while fake data only propagates lies.
— Approval popups proliferate when tool names mismatch across chains; a unified naming scheme, wildcard unlocks, and per-ticket approval memory cut 100+ clicks to 23.
— Context compression loses task anchors (constraints, conversation-only facts); storing entity indices separately from narrative, and requiring conversation-only deliverables, preserves them.
— Cache blocks invalidate on any byte change; pinning stable content and moving variable fields after the fixed block raised cache hit rate from 15% to 99.8%.
— Cost issues show no functional symptoms; only measuring prefix-level cache hit differences reveals them.
Conclusions

The article's core thesis is that agents fail silently because the model faithfully processes whatever it's given, including faulty tool outputs and misleading context; the pipeline is where lies originate.

A pattern emerges: nearly every pit traces back to the developer's own integration code, not the LLM. The author admits shifting focus from "training the model" to "repairing the pipeline", a framing that generalizes to any agent builder.

The nudge approach (inserting a note into context rather than interrupting or retrying) is a clever low-cost intervention, but its effectiveness depends on the model actually reading that note, which itself isn't guaranteed.

The cache-block pit is particularly insidious because it produces no visible malfunction; only cost monitoring catches it, suggesting agent cost observability is a prerequisite for production use.

The 15% to 99.8% cache hit rate improvement shows how small design decisions about content ordering can dominate token costs.

Concepts & terms
Silent failure
A failure mode where a system (here, an agent pipeline) produces no error, crash, or warning, but behaves incorrectly. The agent may stream output, call tools, and report success while the actual result is wrong or incomplete.
Nudge (轻推)
A runtime intervention where a short informational note is appended to the model's context (for example, "you already searched this keyword") to gently steer behavior without interrupting, retrying, or penalizing the model.
Cache block invalidation
In LLM API billing, a large prefix (like system prompt + tool definitions) is cached as one unit. If any byte in that prefix changes, the entire block is invalidated and billed at full price again, even if most of it is identical.
Context compression
When a conversation exceeds the context window, the system compresses older messages into a summary. This can drop task-critical anchors like constraints or facts that only exist in the conversation, causing the agent to forget or violate instructions.
Exit code sentinel
A machine-readable marker appended to command output that indicates success or failure. Requiring it forces the model to rely on a deterministic signal rather than inferring success from free-form output text.
From the discussion
Featured comments
组合搭配函数

It's truly scary when there is no error—the program just spins around doing nothing, and you don't even know where to start looking. The exit-code sentinel trick is solid; that's how you strangle the success hallucination.

See top comments, translated →
Source: juejin.cn ↗ Google Translate ↗ Backup ↗