Agents That Never Error Are the Scariest: 10 runtime pitfalls from 6 months of building a coding agent
theme: channing-cyan
The Scariest Thing Is When the Agent Doesn't Error: 10 "Looks-Normal" Runtime Pitfalls from Six Months of Building My Own Coding Tool
Preface
Starting in June, I started building an AI coding tool for myself: talk to a large model in the terminal, it can read code, modify files, run commands, and work across multiple rounds on its own (similar to Claude Code), backed by DeepSeek. Over six months, 359 commits, and 129 were bug fixes.
After all that fixing, my biggest takeaway is one sentence: when an agent goes wrong, eight times out of ten it won't throw an error.
It sounds counterintuitive. It still streams output, still calls tools, still politely says "done" — but the root is already twisted. Crashes are easy to debug; quiet breakage is hard. This article picks 10 of the most representative runtime pitfalls from these six months and explains them thoroughly: every one looked "normal" on the surface, but digging deeper revealed real problems. First, a quick overview table:
| # | Surface (looks normal) | Truth |
|---|---|---|
| Pit 1 | Tool "spins forever" like a freeze | I forgot to send the "done" signal |
| Pit 2 | Input box cleared = sent? | The message was silently swallowed by both ends |
| Pit 3 | Fluent tone saying "basically done" | Only half the work was done (early wrap-up) |
| Pit 4 | Command failed, model says success | Output looks like success != actually succeeded |
| Pit 5 | Every action is normal | 99 tool calls combined |
| Pit 6 | Guidance worked, still 0 hits | Model copies the erroneous pattern |
| Pit 7 | Tool output rich, format normal | Directory is fake, a three-layer lie chain |
| Pit 8 | Approval popups keep popping, looks secure | Names not aligned, security spinning idle |
| Pit 9 | Long task's "memory" gets worse mid-way | Compression crushed the task anchors |
| Pit 10 | No errors, no lag, everything works | The bill is quietly bleeding |
Pit 1: Tool "spins forever" — not that the model died, but I forgot to say "done"
The surface is scary: a certain tool on the UI keeps showing "running...", the conversation looks frozen, and no matter how long you wait, nothing follows. First reaction: is the model stuck? Is the network down?
The truth is less mysterious. Two types of tools (entering/exiting plan mode) don't go through the normal execution flow. At the start I sent the "started" signal, but at the end I forgot to send the "done" signal. The frontend is an honest worker: it waits for the end signal, and if it never comes, it keeps spinning — the model had already finished talking long before.
Following this thread, I fixed seven kinds of similar "silent failures", all with the same symptom (no output/stuck) but different root causes: missed end signals, error messages swallowed by deduplication logic, message ledger mismatches after session recovery, and state flags not reset causing new inputs to be silently ignored.
Patching the signal is just the surface. The real fix comes in three parts: any exception path needs a fallback catch — even on error, give the user an explanation; during session recovery, reconcile and fill placeholders; put state reset in finally — so failure paths also execute.
In short, when an agent "freezes/no output", first check which sentence in the event stream wasn't spoken, don't blame the model.
Pit 2: The message you sent was never received by the model
During generation, you add a comment "by the way, run the tests too", press Enter — the input box clears, looks like it's sent. When it finishes, it never mentions tests, wraps up according to the original plan. You'd think the model "didn't understand", send it again, still no use.
The truth is a silent conspiracy between both ends: during generation my code directly refused new input, while the frontend cleared the input box first (giving you the illusion of "sent"), and the backend, seeing "busy", simply returned (without a word). Your message evaporated between the two ends — no trace in the UI, and in the model's world it never existed.
The fix is to change "reject" to "queue": new input during generation gets stored, and after this round, at the round boundary it gets handed to the model, so it can follow your addition next round; if queuing also fails, at least pop a message saying "still generating, this message wasn't sent".
In the "clear UI first, then call" pattern, any silent return by the callee equals data loss. The UI feedback must match what actually happened.
Pit 3: Work stops halfway and says "done" — early wrap-up
The model gets a tool result (e.g., background compilation is still running, it only checked half), then writes two sentences "basically done" and ends the conversation. Fluent tone, beautiful formatting, looks like a normal sign-off.
The main loop has a hard rule — "if the model's reply has no tool calls, wrap up". The rule itself is fine, but the reason the model "doesn't want to call tools again" might be because it thinks the work is done. The task clearly has five steps, but it walks two steps and starts summarizing.
Digging deeper, I fixed this pit twice. The first time I added a guard: check whether the ending text contains keywords like "done", "finished", and if not, give it a gentle nudge — insert a note into the context saying "the task might not be finished yet" (this is a nudge). That backfired: the keyword list was too narrow, and when the model wrote "simplified and done" or "everything in place", it didn't match → the guard misjudged it as unfinished → stopped it and let it continue → it wrote again → stopped again... The user saw three or four consecutive endings with no new messages in between.
The second time I switched to a measurable threshold: only if the ending text is fewer than 30 characters does it count as "empty-handed wrap-up" (got a result and only spit a few words before leaving); long summaries were let through. That stabilized immediately.
To judge whether the model has finished or not, don't guess semantics — keyword regexes will never be exhaustive; use measurable signals: length, round count, tool call count.
Pit 4: Success illusion — the command actually failed, but the model said success
This is the scariest kind. The tool output has some content, the model treats it as success, and continues to the next step — thinking tests passed, it makes a commit; thinking dependencies installed, it starts packaging. Throughout, it's confident and logically coherent, like a properly working agent.
I hit both variants. First: the command-running tool only showed the screen output to the model, without the exit code — the model judged success based on whether the output looks like success, but whether output looks like success and whether it actually succeeded are two different things. Second variant, more unjust: on timeout, exceptions, and other failures, the error text is free-form, and another piece of my code guessed success/failure by "whether the output prefix looks like success" — guessed the wrong direction, turning failure into success.
The fix is to let the machine judge, not have the model read minds: force command results to carry an exit code sentinel; without a sentinel, count as failure (better to over-kill); converge success/failure determination to a single exit point, uniformly producing structured results.
Facts fed to the model must be machine-determinable. An ambiguous success will make it pile error on error without ever knowing.
Pit 5: Model treats search tool like a dictionary — each action normal, combined a disaster
Observe an execution of a medium-sized requirement: read files, search code, make a small change, look again, search again... each action individually is completely normal — that's how an agent should work. But one task used 99 tool calls, while the reasonable value is one-third of that, burning triple the tokens.
I ran behavioral statistics on months of real execution records: 20% of rounds were wasted — repeated calls with the same parameters (387 times), repeated searches of the same keyword (347 times), back-and-forth of modify-and-look (98 times). The model has no habit of "I checked this before"; each time it treats it as the first time.
Two fixes, neither relying on interruption. One: set rules in the prompt — "read everything first, then act; don't treat search as a dictionary" and "plan changes in batches, do it in one pass". Two: runtime nudge — when the same file is read for the 3rd time, or the same search term for the 2nd time, insert a note at the end of the context: "you just checked this, the answer is in history". No interruption, no retry, no punishment, like a colleague lightly tapping you; modified files automatically reset their counters (re-reading after modification is legitimate).
Every action alone is correct, combined it's a disaster — such problems only show up in statistics, never by watching a single execution.
Pit 6: You give it a syntax name, it copies the erroneous pattern
After a web scraping failure, I wrote guidance in the failure message: "try web_search with site: syntax". The model immediately followed — the guidance worked — but the result was still 0 hits, then it spun idly for two more rounds, called tools 5 times, and finally concluded "the content isn't in the index", a wrong conclusion.
The model stuffed the entire failed URL into site: (site:zhuanlan.zhihu.com/p/37196599) — but site: only accepts a domain. It didn't understand the guidance; it was imitating character by character: the guidance had no example, so it took the most prominent material in the recent context (the failed URL) and fabricated a form on the spot. I ran an A/B comparison: correct form site:domain title words gave the target as the first result.
Fix: guidance given to the model must include a copyable template — "pre-assembled prefix + blank slot" (site:${domain} page title keywords) — that's ten times more effective than explaining grammar. A guard test pins down both forms: output must contain the correct form and must not contain known misused forms.
The model is a master copier, not a grammar student. You give it a syntax name, it fabricates a form; you give it an example, it replicates.
Pit 7: The tool lied to the model, the model lied to you — a three-layer lie chain
Ask the agent to look at some external directory, it lists a bunch of files — rich output, normal format, everything looks smooth. Then it starts a full recursive search, hits a wall, and solemnly tells you: "this repository isn't here."
Investigation shows it was the tool's fault: list_dir had a parameter that was silently ignored, and instead of erroring, the tool "kindly" fell back — it returned the directory tree of the current workspace to the model. The model doesn't know the directory is wrong, carefully reasons with the fake map, produces a fake conclusion, and solemnly reports it to you.
The whole lie chain is three layers: tool's well-meaning fault tolerance (source of lying) → model reasons on false information (passes the lie) → outputs false conclusion (user deceived). The most ironic part: each layer alone looks "working normally" — the tool returned data, the model gave a conclusion.
The fix is to forbid silent fallback at the tool layer: if a parameter isn't received, explicitly error out. Better to let the model see failure than hand it a well-meaning replacement of fake data. In addition, nail the "correct route to find the repo" as an example in the guidance (see Pit 6).
Well-meaning fault tolerance at the tool layer is the source of lies. Explicit failure beats silent replacement — the model can self-heal when it reads the error text, but can only self-congratulate when it reads fake data.
Pit 8: Over a hundred approval clicks—that's not security, it's a bug
During a browser automation task, approval cards popped up one after another, clicked through one by one — counted on the actual machine, over a hundred times. My intuition at the time: the tool is dangerous, and the many approvals mean the system is seriously checking.
The unlock list was clearly configured, so why did it block every time? Investigation revealed a name mismatch: in the approval step, the tool name used was one form (bare name mcp_call), while the unlock list registered another (full name mcp__service__tool). Each link in the chain was "normal" on its own, except the names didn't align → the list never matched → every time a card popped. The security system looked diligent, but was spinning idle.
Three-part fix: unify the name across all approval chain links; support wildcards in the unlock list (one rule unlocks all tools of a service); remember tools you've approved in the same ticket and don't ask again. A hundred-plus clicks dropped to 23.
More approval popups doesn't equal more security. Every "extra confirmation" trains the user to click "Allow" mindlessly — that's the real danger.
Pit 9: Running long tasks, the agent "loses memory" mid-way
On a long multi-round task, in the latter half, the agent starts repeatedly asking things already confirmed, or drops early constraints ("don't touch that file" — it touched it). It looks like the model is weak, bad memory.
The context window is finite; once it exceeds a threshold, old conversation gets compressed into a summary. The problem lies in the compression action itself. I ran a controlled experiment: when the narrative in the summary was dropped, task quality degraded significantly (the model violently rummaged through archives to retrieve the original text); when the entity index was dropped, nothing major happened (it could still retrieve without relying on the index). So what compression loses isn't text, but task anchors: earlier constraints, facts that only exist in the conversation (e.g., "after completion, write the result as traceability code X"), which don't land in any file, and evaporate once compressed.
Two fixes: first, the summary slot is made dual-segment — the entity index is deterministically extracted from the original text by code, never goes through the model's head (won't be rewritten during compression), and only the narrative part is allowed for the model to converge; second, the most critical facts in the task are deliberately written as "conversation-only deliverables" — explicitly state they exist only in this conversation, not in any file, and require "deliver after everything is done" to prevent early handoff.
When long tasks deform mid-way, check the compression records first, then the model. Anchors must be designed to be unbreakable by compression.
Pit 10: The quietest pit — the bill is bleeding, everything looks fine
No errors, no lag, no functional anomalies, the assistant does the same work every day. The only change is the bill quietly rising. On investigation: new sessions' cache hit rate was only 15% — most of the fixed overhead was being recomputed at full price repeatedly.
Four controlled experiments revealed that the cache's minimum billing unit isn't per-character, but a large block of "fixed content" at the front (tool documentation + system prompt, about 11k tokens) packed as one block — even one byte different inside the block, and the entire block is invalidated and recomputed at full price. And my code appended things that change at the tail of this block: reply language, skill list, memory directory... skill refresh or language change, and the whole block was invalidated.
It's like a whole-box discount: once opened, it doesn't count as whole-box price — and I was opening it every few minutes.
Fix: pin down "what must stay stable" (directory sorted by name, injected content byte-identical), and move "what changes" entirely after the block. After the fix, on the same machine, rerunning the same question, the hit rate went from 15% to 99.8%.
The most expensive kind of silent pit: no symptom at all, only the bill. Cost issues require fingerprint forensics (the difference in hit rate between two requests with the same prefix); watching functional tests will never find them.
Conclusion
Talking through this, the title's sentence lands: an agent that doesn't error is the scariest.
Looking back, none of these 10 pits is "the model isn't smart enough." The model absorbed all the information it saw — including the wrong version I handed over. Spinning was because I didn't send a signal, message swallowing was because I made the UI lie, hallucination was because I didn't give an exit code, copying was because I didn't give an example, popup spam was name mismatch, memory loss was my compression losing anchors, and burning money was me putting changeable things into the cache block.
These six months, my work focus gradually shifted from "training the model" to "repairing the pipeline": signals sent completely, facts machine-determinable, failures explicit, anchors unbreakable. The model is the part that always works hard; the pipeline is the part that quietly lies.
The complete pit ledger (14 chapters, indexed by topic, including fix details for each pit above and more environment-related pits) is in the repo: deepseeker-code → docs/经验汇总.md. If you've stepped on the same pitfalls, feel free to open an issue to discuss.
Top 1 from juejin.cn, machine-translated. The original thread is authoritative.
It's truly scary when there is no error—the program just spins around doing nothing, and you don't even know where to start looking. The exit-code sentinel trick is solid; that's how you strangle the success hallucination.