Don't Blame the Prompt: Trace the First Mismatch in Your Agent's Execution
🔥🔥【封闭结束10】Agent 回复准确率优化与问题定位:别一出错就改 Prompt
From the moment large models became deployable, I have been working on landing them: RAG, AI Coding, workflow, harness, automation, workbenches, and also MCP and Skills. Some projects ran stably in business, others stopped halfway. I often hear: "The model determines Agent capability." I think that's only half right. The model determines the upper bound of capability; whether it can be used reliably and whether errors can be traced back to their cause depends more on the entire Agent engineering suite (also called harness in the industry, meaning the runtime and evaluation shell around the model) — sessions, tools, memory, permissions, Traces, regression testing. The "Enclosed End" series has ten articles; this is the 10th and final one. When users feel the answer is "wrong", it is often not that the model suddenly got stupider, nor necessarily that the Prompt was poorly written, but that somewhere between understanding the user, remembering context, calling tools, and generating the answer, a step has already gone out of alignment. What we need to do is first use the execution chain (Trace) to find the first point of mismatch, then decide whether to change the session, the tool, or the Prompt, and write the case into regression tests. I hope that after reading all ten, you have a complete thread of "how production Agents run".
Continuing with the series' case OpsPilot (enterprise operations assistant, fictional). Online feedback came in:
用户:"明明说的是生产环境,怎么查成测试环境了?"
用户:"明明说的是生产环境,怎么查成测试环境了?"
The team's most common reaction is to open the main Prompt and add a few more constraints. It might improve that day, but two days later the user rephrases it and the problem appears again. This ultimately leads to: the Prompt gets longer and longer, yet still can't explain where exactly the error lies.
The root problem is that the user only sees the final reply, while there are several steps before it — intent understanding, session memory, tool parameters, knowledge retrieval, and so on. Any step going wrong can appear as "the model answered incorrectly".
Let's trace through the user's problem report from the beginning to see where the issue is. Then we'll look at how to investigate problems with L1–L5, fix common problems, check hallucinations, and finally write cases into regression tests.
How to investigate this issue feedback: open the Trace first
The on-duty colleague used traceId to pull the complete execution record. To clarify the problem, you must see what the Agent actually did, not just the final reply in the user's screenshot.
Trace (execution log): A structured record of a single Agent execution, containing at least the user input, session context, intent/tool call parameters, tool return summaries, and key claims in the answer with evidence references. Use
traceIdto string a user request from UI to model and tool. When investigating, look at what record each step left, not just the final reply.
Below is a simplified OpsPilot Trace (the same class of online problem as "production queried as test" at the opening of the 【Enclosed End】 series, with the answer claim fields completed here):
const trace = {
traceId: 'tr_ops_001', // unique id of a request; use traceId to pull the full chain during online feedback
input: '查一下生产环境 order-service 的错误日志', // user's original wording
context: {
tenantId: 'tenant-acme',
currentEnvironment: 'prod', // current environment recorded in the session
},
intent: {
type: 'query_logs',
environment: 'prod', // intent recognition result is still production
},
toolCalls: [
{
callId: 'call_1',
name: 'query_logs',
args: {
service: 'order-service',
environment: 'test', // first obvious deviation: parameter filled as test
},
status: 'success', // interface returned success, but that doesn't mean the environment is correct
},
],
answerClaims: [
{
text: '以下是生产环境日志',
evidenceCallIds: ['call_1'], // claims to rely on this query, but it actually queried the test database
},
],
}
What does this Trace expose?
- The user explicitly asked to query the production environment.
- The environment in the session state is
prod. - The intent recognition result is also
prod. - When the tool call is actually made, the parameter becomes
test. - Although the tool executed successfully, it returned data from the wrong environment.
- The final answer describes it as production environment logs.
Therefore, the first obvious deviation is at L3: tool parameter generation or passing stage (intent and session are both correct; the error lies in the parameter filled when actually calling the tool). The final sentence still says "production environment logs", which looks like hallucination at L5, but actually the parameter was already wrong earlier; without looking at the Trace, it's easy to change the RAG or switch the model.
Teams often take detours; a comparison between the common mis-fix and a more stable fix is as follows:
| Easy mis-fix | More stable correct fix |
|---|---|
| Add "must use production environment" to the main Prompt | Find at which step the parameter changed from prod to test; add consistency check in Tool Runtime before sending the request |
| Switch to a stronger model | In this case the intent is already prod; changing the model often reproduces the same parameter error |
| Touch vector database / RAG first | This request did not go through retrieval; L4 can be skipped |
| Only look at the final natural language | Use structured Trace to compare context / intent / toolCalls[].args |
status: "success" only means the tool call completed successfully; it does not mean the environment was correct, nor does it mean the answer is correct.
A small piece of deterministic code can turn "first parameter deviation" from manual item-by-item comparison into an automatically checkable logic. The example below uses the same fields as the Trace above; if your online logs have a different structure, replace with your own field names:
// Fields match the Trace example above, convenient for automatic checks and regression cases
type OpsTrace = {
context: { currentEnvironment?: string }
intent: { environment?: string }
toolCalls: Array<{ callId: string; args: { environment?: string } }>
}
// For troubleshooting: find the first tool parameter inconsistent with session/intent
function findFirstEnvironmentMismatch(trace: OpsTrace) {
const expected = trace.context.currentEnvironment ?? trace.intent.environment
if (!expected) return null
for (const call of trace.toolCalls) {
const actual = call.args.environment
if (actual != null && actual !== expected) {
// Return the first incorrect parameter to mark it as 'error at tool layer'
return { layer: 'L3' as const, callId: call.callId, expected, actual }
}
}
return null
}
// Before actually calling the interface: block if environment doesn't match session, don't show success while calling the wrong environment
function assertEnvironmentBeforeCall(
args: { environment?: string },
ctx: { currentEnvironment?: string },
) {
if (!args.environment || !ctx.currentEnvironment) return
if (args.environment !== ctx.currentEnvironment) {
throw new Error(
`environment mismatch: args=${args.environment} context=${ctx.currentEnvironment}`,
)
}
}
Running the above check on tr_ops_001 gives a clear conclusion: both session and intent are prod, but the tool parameter is test. The same user input should be written into the regression case env-mismatch-001 (see regressionCases third item in "Write cases into regression tests" below): when querying production, test must not be passed to query_logs.
The problem is at L3, so modify it like this:
In Tool Runtime, call assertEnvironmentBeforeCall: if the environment doesn't match, don't call the log interface, write the reason back to the conversation, and don't let the interface return success while querying the test database.
When the session environment is already prod, the final parameter should be merged using session/intent in code, rather than copying the environment from the model's returned JSON. Only if the user explicitly says to change environment in this turn, use that sentence to override.
// Before calling the tool: merge the model-filled parameter with the session-confirmed environment into the final parameter
function mergeLogQueryArgs(
modelArgs: { service?: string; environment?: string },
ctx: { currentEnvironment?: string },
intent: { environment?: string },
) {
// Prefer session/intent-confirmed environment, to avoid the model JSON accidentally filling in test
const settled = ctx.currentEnvironment ?? intent.environment
return {
...modelArgs,
environment: settled ?? modelArgs.environment,
}
}
Even if the request has already been sent, check which environment the tool return indicates. If it's actually test database data, the reply must not say "production environment logs" — either describe the result according to the real environment, or clearly tell the user: this query did not get production.
After the parameter merge and pre-request validation are fixed, run the user's original sentence from the opening once more; and add this scenario to the regression case env-mismatch-001 (defined in "Write cases into regression tests" below) to run before release.
Production Traces should also include Prompt/model version (whether a recent Prompt change made it worse), tool return summaries, and parameter validation results; fields should be structured and searchable. When credentials or sensitive data are involved, desensitization and access control cannot be omitted.
Investigate problems by layers (L1–L5)
At which layer is the error (L1–L5)
L1–L5 are the layering notation used in this article; they are not an industry standard; their value is to let the team speak the same language for the same online problem. They roughly correspond to previous articles: L1 SSE/HITL and front-end protocol, L2 session and authentication, L3 ReAct and Tool Runtime, L4 RAG, L5 generation and Prompt.
| Layer | Scope of investigation | Common problems |
|---|---|---|
| L1 | SSE and HITL | Streaming event loss, cancellation ineffective, human confirmation not displayed |
| L2 | Session and permissions | History truncation, identity context missing, tenant crossover |
| L3 | Orchestration and tools | Intent error, tool selection error, parameter error, loop not stopping |
| L4 | Retrieval and knowledge | Missing recall, wrong recall, stale documents, version inconsistency |
| L5 | Model and prompt | Evidence correct but fabrication, format failure, excessive refusal |
Users say "inaccurate", common symptoms can also be classified into six categories: intent error, context loss, tool/parameter error, same-name resource selection error, retrieval evidence error, expression beyond evidence. Different root causes require different fixes; the Trace example above belongs to "tool/parameter error + expression describes wrong data as the correct environment".
Recommended investigation order: find the first mismatch first
When you get an online problem, scan from L1 downward along the real trajectory (skip layers not involved; for tool-type tasks, focus on data before and after the call):

Image meaning: check the Trace in sequence from L1→L5; once a layer is confirmed wrong, it's not necessary to go through the remaining layers; you can directly fix and add a regression. Skip L4 if RAG wasn't involved.
Stop at the first place that doesn't match expectations. If uncertain, re-run the same user input and compare. If tool parameters are already test and the tool returns test logs, but the answer says "production logs", check L3 first; if the tool returns production logs but the answer describes them as test, check L5.
Three common scenarios, how to fix each?
Case 1: User asks for a feature explanation, but Agent calls a real-time tool
用户:"日志查询是做什么的?"
This sentence asks what the feature does. It should query the knowledge base or answer with text, not call the real-time log interface. But the Agent called query_logs and asked the user to pick an environment.
First point of investigation: intent recognition and tool routing at L3.
First divide user speech into two categories, so routing won't be tangled:
| What the user wants | What to do | This example |
|---|---|---|
| Ask for explanation (what is the feature, how to use) | Query knowledge base or answer with text, don't call query_logs |
「日志查询是做什么的」 |
| Actually query (give me the current situation) | Only then call real-time tools like query_logs |
「查一下生产环境支付服务最近的错误日志」 |
| Easy mis-fix | More stable correct fix |
|---|---|
| Pile "see 'what is', don't call tool" into Prompt | Make intent classification according to the table above; write boundaries into tool description: query_logs only queries real-time logs, not explanations |
| Use keyword rules and hope | Write both sentence patterns from the table into regression tests; like "帮我查下日志" which has an execution tone but doesn't specify environment or service, ask first, don't force a tool call |
Tool description shouldn't just say "query logs" — the model doesn't know that "what is" type questions shouldn't call it. In LangChain, you can use tool()'s description to write it clearly (illustration, same writing style as the practical article):
import { tool } from '@langchain/core/tools'
import { z } from 'zod'
// Write into the tool description when it can and cannot be called; more effective than stacking main Prompt
export const queryLogsTool = tool(
async ({ service, environment }) => {
// Here call your log query interface and check user permissions
return JSON.stringify({ service, environment, lines: [] })
},
{
name: 'query_logs',
description: [
'Call when the user has explicitly asked to query real-time error logs for a specific service and environment.',
'Requires service, environment.',
'Do not use for explanation-type questions like "what is log query" or "how to use"; answer those with text or knowledge base, do not call this tool.',
].join(''),
schema: z.object({
service: z.string().describe('Service name, e.g. order-service'),
environment: z
.enum(['prod', 'staging', 'test'])
.describe('Environment'),
}),
},
)
You can also give the two intent types names in code (any name is fine, just keep team consistency), for example knowledge_explain (explanation) and realtime_action (execution). After fixing, add to regression cases (e.g., intent-001) to lock the behavior.
One sentence: when the user just wants to understand a feature, just answer; don't pull out the "go to the backend and pull logs" routine.
Case 2: "Where are its logs?" — context loss
用户:"查一下生产环境的 order-service"
Agent:"当前有 3 个实例……"
用户:"它的错误日志呢?"
Here "它" refers to the previous turn's order-service. Normally, it should continue to query the error logs of this service; if the Agent asks for the service name again, or queries another service, it's likely that the previous turn's conversation wasn't carried into this turn.
First point of investigation: session context at L2.
| Easy mis-fix | More stable correct fix |
|---|---|
| Blame the model for "not understanding what 'it' refers to" | Use Trace to see whether order-service is actually in the chat history sent to the model in this turn |
| Endlessly extend history messages | Separately record "which service and environment are currently being queried" in the session (see ConversationState below) |
Sometimes it's not that the model is dumb, but that the message sent is too short. After window truncation, the model might only receive the second sentence:
// Bad example: if only this sentence is actually sent, the service name from the previous turn is lost
const messagesActuallySent = [
{ role: 'user', content: '它的错误日志呢?' },
// The previous 'query production order-service' didn't come in, so 'it' can't be matched
]
You can choose one of two fixes, or use both:
- Simple approach: when composing the Prompt, include the few turns relevant to the current question verbatim (enough when the conversation isn't long). In LangChain / LangGraph, don't reduce the second turn to only the last sentence, illustration:
import { HumanMessage, AIMessage } from '@langchain/core/messages'
// All turns' original dialogue must go into messages; only passing the last sentence causes 'it' reference loss
const messages = [
new HumanMessage('查一下生产环境的 order-service'),
new AIMessage('当前有 3 个实例……'),
new HumanMessage('它的错误日志呢?'),
]
await agent.invoke({ messages }) // When running Agent with LangGraph, also pass multiple turns
- More stable approach: maintain "current service, current environment" on the server side, not relying on the model to find it from history. For example:
// On the server, remember