跪拜 Guibai
← Back to the summary

Six Ways a Production Agent Dies: A 200K-User Project's Four Hard-Won Refactors

The 6 Ways an Agent Dies After Launch: A 200K-User Project's Four Rewrites

We built a business-facing Agent for end users, serving 200,000+ users with 500,000+ daily conversations. We rewrote it four times, killed about 90% of hallucinations, and dropped P95 from 15s+ to under 10s. This post doesn't brag about results. It tells you how each version screwed itself, and what we understood after each death.


0. First, the conclusion: none of the four versions were designed

If you are shipping an Agent, remember this: none of the four versions was "designed." Every one was pushed forward by the problems the previous version exposed.

We were solving the same set of contradictions the entire time:

These four pull against each other: less hallucination demands more constraints and more recall, so tokens go up and latency goes up; faster means less reasoning, so hallucinations go up; easier maintenance means splitting prompts, but then the segments fight each other.

You can tell whether an Agent team is mature by whether they can articulate where the contradiction is in this version. If they can't, they're probably still on version 1.

Figure 1 · Four versions of evolution: each one was pushed by the previous one

Figure 1 · Four versions of evolution: each one was pushed by the previous one


Death #1: Heavy reliance on intent recognition — every new business meant retraining the algorithm

The start was small

In November 2024, V1 launched.

The origin was plain: we pulled the top 10 daily customer-service questions and found a huge number of "why was my order flagged as abnormal." Users didn't know the basis for the judgment and kept asking.

The intention was just to turn my own repetitive work into an SOP for users — not to build a general-purpose assistant.

Architecture

A pre-stage intent recognition layer routed to the corresponding tool or subagent based on static intent.

user query → intent recognition → routing → tool/subagent → reply

It did one thing right

It got the Agent's front-end/back-end SSE streaming pipeline working.

That value is seriously underrated. Streaming is the foundation of the whole experience — users see characters appear one by one while waiting, rather than staring at a spinner for 15 seconds. Many teams patch this in last, and then have to redo the entire timeout budget for the chain.

It died on three things

  1. No multi-turn capability — the second question lost the context
  2. Heavy dependence on intent — if the intent was misclassified, everything downstream was wrong, and confidently so
  3. Every business integration required the algorithm to retrain parameters — delivery cycles measured in weeks

The third was fatal. Every new scenario meant scheduling algorithm time, labeling data, training a model, and evaluating results. The business side couldn't wait, and we couldn't sustain it.

Why we later dropped the traditional classification model

Because that's what V1 did.

The problem with a classification model isn't accuracy; it's that it can only say "which category," not "which specific order. " It can't handle "this order," "the last one," or "the one I mentioned just now." And every new scenario required retraining the algorithm.

The only benefit is that it can output observation metrics per intent, which helps the business collect revenue data. In other words, a classification model gives you observability, not capability.


Death #2: Stuff everything into the LLM — tokens went from 3,200 to 12,000

Why we rewrote

In 2025, models got noticeably stronger. We saw that the assistant was very stupid: it read but didn't understand, gave the same answer for multi-turn inputs.

We decided to rewrite. Core idea: don't rely heavily on intent recognition, let the LLM handle the whole reasoning and decision-making, and support multi-turn.

Architecture: looks perfectly standard

Gateway layer   → auth, security checks, save conversation
Context layer   → user info, ticket info, anomaly info
Orchestration layer → RAG retrieval
Capability layer → tool, skill
core agent      → ReAct loop reasoning
Model layer     → cloud vendor model service / internal agent platform
            + monitoring and evaluation

This architecture would pass scrutiny in any Agent architecture article.

It died on one sentence

We dumped all tools and all context into the core agent in one go.

The result was inevitable:

But the most fatal part wasn't these

It was that nobody dared touch the prompt.

All business shared one prompt. Change one part, and every business had to re-review before shipping.

One prompt became the most feared file in the company. Change one character, go through three rounds of review, pull in five business stakeholders. In the end, people preferred patching the code instead of the prompt — because a patch at least didn't affect everyone else.

Root cause

One sentence: we mixed "choosing capabilities" with "executing capabilities."

Having one model both decide "which tool to use" and "how to use the tool to complete the task" made it flip-flop between the two goals. The larger the candidate pool, the more it flip-flopped.

Figure 2 · V2's architecture was flawless on paper, died from stuffing everything in

Figure 2 · V2's architecture was flawless on paper, died from stuffing everything in


Death #3: Adding an orchestration layer — the pressure just moved house

What we did

Added an "orchestration" layer before the core agent: packed tool description corpus into a knowledge base, used a small model to pick capabilities based on the user query, then handed the narrowed set to the core agent for LLM reasoning.

user query → orchestration (small model picks capabilities) → core agent (LLM reasons)

Result

Capabilities were indeed decoupled. The core agent got a cleaner tool list, and the prompt no longer had to include all tools.

But the problem didn't disappear — it moved from core agent to orchestration layer:

Lesson

Moving complexity forward isn't solving it; it's just relocating it.

After that version we realized: the real solution isn't "who picks the tools," it's how to make the candidate pool small enough and the selection deterministic enough.


Death #4 (the turning point): Layered governance, stability first

V4's core words: layered governance, stability first, large-scale noise reduction, hallucination control, cost control.

This version didn't change the architecture; it changed where the constraints live.

Specifics involve many internal designs — how tool descriptions are graded, how recall strategies are configured, how much orchestration should converge — this is my bread and butter, so I won't detail each one. But the thinking can be explained, and thinking is more valuable than parameters:

  1. The same tool information shouldn't be maintained in just one version. Retrieval needs "detail" to compute similarity accurately; selection needs "concise" so it doesn't blow up the context. Using one corpus to serve both is inherently contradictory — you have to split by purpose, and make both sides align.
  2. Changing configuration shouldn't require a release. Tool description text is an operational asset, not code. It must be hot-updatable, or every description tweak needs a release window, and "operability" is always a negative score.
  3. The candidate pool must be narrowed before model reasoning. Not because the model can't choose, but because the candidate pool itself is the breeding ground for hallucinations — the larger the pool, the more the model tends to "guess at something that looks right."
  4. Memory can't be used as-is. We stepped on this for a long time before we got it; more below.
  5. Deterministic things should never go to the model. Fully deterministic chains get hardcoded returns; semi-deterministic ones (like extracting an order number from a conversation) don't go through reasoning either, but go through routing. The model only handles genuinely uncertain parts.

Why memory can't be used as-is

A concrete example. An anomaly list — the tool returns a big JSON. Previously we saved the JSON verbatim into memory. Three things happened at once:

There's only one fix: when storing, don't store the raw structure. Concatenate the fields into a semantic summary and store that.

One move solves three problems: prevents hallucinations, saves tokens, aids coreference.

Results

Note: "about 90%," not "eliminated." I'll say what's left later.


Hallucination isn't one problem; it's six

Most people say "don't fabricate" when talking about hallucinations. Useless.

Because hallucination isn't one problem — it's six. The root causes are different, and so are the remedies.

# Type Behavior
1 Skipping tool Should call a tool, but answers directly from memory, returns immediately
2 Fabricating parameters No anomaly order found, but there's a number-like type in context, so it assigns it as the order number (ticket ID, user ID)
3 False success Tool call fails but reports success — the report API returns an error, but it returns "reported successfully"
4 Cross-turn contamination Treats A order's diagnosis as B order's
5 Cross-object contamination Same, pollution across business objects
6 Routing hallucination Calls a tool that doesn't exist at all

This table is worth printing and pasting on the wall. Because many teams are using "add forbidden words" as the one cure for six different diseases.

Figure 3 · The six hallucination entry points: they live in different places in the reasoning chain

Figure 3 · The six hallucination entry points: they live in different places in the reasoning chain

There's an order to governance, and it can't be reversed

Architecture layer: strip deterministic chains, early exit, shrink candidate pool

Input constraints: schema rules, parameter format constraints, memory summaries, forced tool whitelist

Process control: orchestration layer pre-narrows the tool set

Bottom safety net: continuous evaluation (real-time sampling + offline replay)

The order matters: first make it impossible for hallucination at the architecture level, then tighten the opening with input constraints, then limit the choice space with process control, and only then back it up with evaluation.

Many teams do the reverse — they go straight for prompt forbidden words. Forbidden words can suppress "tone," but not "facts."

Figure 4 · The order of four governance layers: the deeper you go, the cheaper; the higher, the more expensive

Figure 4 · The order of four governance layers: the deeper you go, the cheaper; the higher, the more expensive

Three actions that actually worked

One: make the model see examples of "using a tool to complete a task" in context.

Initially, our memory stored only results, not tool calls, because we thought it was redundant with the model's output. Result: the model learned to just reply in text — it never saw any precedent for "getting things done via tools," only "replying with words."

This is a classic pit: you train it to see, so it becomes what it sees. In scenarios where tools should be called, the memory must retain the traces of tool calls.

Two: write extraction rules into the parameter schema.

For example, anomaly order numbers: how many digits, where to find them in context, where to find them in historical memory. The model doesn't know your ID format — if you don't write it clearly, it will make one up to fit.

Three: make the candidate tools basically fixed before reasoning starts.

Which tools can be used in the ReAct stage should ideally be decided at the orchestration stage. If during reasoning a tool is wanted but not in the candidate pool, the correct response is to retry or ask, not to let it through.


Early exit: the biggest step in hallucination reduction isn't making the model smarter

ReAct's rhythm is: model → call tool → back to model → decide again.

But some tool calls return a terminal result; no further model reasoning is needed.

For example, pushing a processing-result card to the user — the returned result is already the final state; there's no need to feed the subagent result back into the model to think about "how to reply."

At that point, end directly: saves tokens, and one less chance for hallucination.

Combined with deterministic chain stripping, this is the main source of "90% hallucination reduction, 15s→10s."

It's not about making the model smarter; it's about not letting the model do things it shouldn't do. I consider this the guiding principle of Agent engineering.


Evaluation: how to prove the judge itself didn't misjudge

One principle

What rules can judge, don't give to the model; what you give to the model must have anchors, reference answers, evidence, and be verifiable.

Split into three layers per this principle:

Layer Scenario Practice
L1 Rules can judge Hardcode the check, don't go through the model. E.g., called a tool that shouldn't be called, didn't report something that should be reported
L2 Needs semantic understanding Check against a reference. Must give the judge anchors and tool-return evidence, no free play
L3 The judge itself is unsure Abandon single-shot judgment, use repeated sampling to converge

Evaluation will always overfit

The judge follows the standard answer, scores get higher and higher, and at launch it falls apart.

This is the sneakiest failure mode of an evaluation system — it doesn't error out; it just makes you more and more confident.

The only fix is structural: when building the evaluation set, tag by intent domain to reduce cross-domain interference; the data used for diagnosis and the data used for final acceptance must be two separate sets, physically non-overlapping.

If you use the same data for diagnosis and scoring, the model evolves toward that data, looks more and more accurate, and learns nothing.

Current state

In evaluation, humans still make up 80% . That's our most unglamorous number right now, and the next thing to tackle.


Self-evolution: plan written, but not yet implemented

We want to build a closed loop: log collection → automatic grading → layered diagnosis → prescribe fix → human review → generate new version → re-evaluate → canary → real-time traffic validation.

I won't detail the chain (it's not running, and writing it would be empty talk). But I can give one judgment now:

The hardest part of self-evolution isn't "finding problems," it's "changing safely."

A system that can automatically modify prompts, if it can't quickly roll back when it breaks something, is a liability, not an asset. So every link in this chain must be able to brake — that's the real reason it hasn't shipped yet.


Directly editing the prompt — aren't you afraid of breaking it?

Afraid. But fear doesn't mean don't edit; it means add constraints to it.

The premise is that the prompt must be concatenated in segments, not one giant blob. Role, boundary rules, tool descriptions, historical memory, real-time context, forced injections — each segment has a clear responsibility, so problems can be localized to a segment.

Under that premise, a few hard rules:

The last one is the most counterintuitive and the most effective: give the system a "cooling-off period." People in a hurry will keep making random changes, the more they change the worse it gets, and eventually no one knows which version broke it.

Key realization: don't treat the prompt as code; treat it as configuration + experiment.


What to do when a tool call fails: not an abnormal termination

All tool calls are wrapped uniformly, catching exceptions, throwing custom exceptions (with the tool-returned error code). Then handle two cases:

Retryable (timeout, network jitter, IO exceptions): automatic backoff retry a few times, if still failing return a fallback. These retries don't consume model resources, because retry doesn't need the model.

Non-retryable: feed it back to the model as a tool response, and let the model decide — change parameters? Switch tools? Or fallback reply.

This is the essence of ReAct: failure is information, not a terminal state. What you need to do is correctly express the failure to the model, rather than making the decision for it or crashing it.

Subagent exceptions are handled the same: catch globally but don't throw, convert to a structured failure signal, and the main agent decides whether to switch domains, ask, or fallback.

One more layer outward: final safety nets like a turn cap and a thinking-timeout cap; when hit, output fallback text, with monitoring alerting and degradation strategies. This layer isn't to improve effectiveness; it's to guarantee the system never hangs indefinitely.


Memory: not about storing as much as possible

A few conclusions, no implementation details:


Problems that still exist (we don't pretend to solve them)

  1. RAG recall's false-positive and miss rates — if recall is wrong, even everything downstream correct is useless
  2. Hallucinations still exist, just much smaller than before
  3. Evaluation is 80% human
  4. Self-evolution isn't implemented yet — the plan is written, but it's not running

I write this not to seem honest. It's because an Agent team that says it has solved everything usually has only built a demo.


Next direction

We're trying a more "deterministic" route: not using a generative model for filtering decisions, but using deterministic filtering + scoring + multi-select, outputting only within a given candidate pool, scoring only without writing answers. Cost and latency are an order of magnitude lower than an LLM, and it's type-safe.

Used in two places: a post-recall filter for "does the query match the tool," and a multi-select within the narrowed candidate pool. The original orchestration model degrades to a fallback, handling only low-confidence and complex queries.

Results aren't validated yet; after validation, I'll write another post. I'd rather give my judgment first, then the conclusion — the reverse becomes selling a solution.


Finally: the real methodology of all this

Looking back at four versions, the real progress isn't any technical choice, but learning to surface uncertainty and grade it:

This thinking explains three seemingly unrelated modules: hallucination governance, evaluation judging, and self-evolution.

Figure 5 · Surface uncertainty and grade it

Figure 5 · Surface uncertainty and grade it

If I can leave one sentence to teams building Agent deployments: don't ask "how to make the model smarter," ask "what exactly requires the model to do this?"


About the details

Some will ask: how exactly to grade tool descriptions? What are the recall thresholds? How does judging vote? What's the word limit for prompt edits?

I really left those out. The reason is simple: these numbers are what I've validated in real business. Outside that scenario, copying them is probably wrong.

If you're working on something similar and stuck in any of hallucination, cost, latency, evaluation — send me your specific scenario, and I can give my judgment directly, which is more useful than a parameter table.


Done: built an end-user AI Agent "Business Assistant" from 0 to 1 (200K+ users / 500K+ daily conversations), a business operations low-code platform from 0 to 1 (new scenarios from week-level to day-level), and funding-safety with 5x peak and zero capital loss.

I'm doing enterprise AI Agent consulting; message me if you have needs.

Comments

Top 1 of 2 from juejin.cn, machine-translated. The original thread is authoritative.

张小妙

Hmm, feels like a sudden enlightenment.

vaskilab

Thanks for the support.