Agent · Artificial Intelligence · Interview

Production Agent Intent Recognition: Route on Validated Structure, Not Keywords

By Sailing · Oct 23, 2026

Read original on juejin.cn ↗ Google Translate ↗ Alt translation

LLM outputs are untrusted until validated, and this pattern makes that concrete for conversational agents: a misrouted intent can turn a log query into a restart in production. Adopting it means agents stop guessing meaning in code and start enforcing schema, permissions, and risk gates at the boundary.

Summary

Intent recognition in production agents is becoming a routing discipline: the model reads the user's sentence and emits a four-part structured result, and engineering code checks that result before anything runs. The concrete pattern, demonstrated in the OpsPilot operations assistant with TypeScript, Zod, and structured output, splits the work into intent, mode, slots, and missingSlots. The model chooses the intent and fills parameters; code verifies format, checks the intent against an enum, enforces allowed modes, and blocks unauthorized or risky results.

Keyword matching gets rejected as a primary classifier because negation, anaphora, ellipsis, and synonyms all break it. Instead, every intent definition carries a description, positive and negative examples, required slots, and allowed modes. Intent and tool are decoupled: one intent can require several tool steps, one tool can serve multiple intents, and compound labels such as 'query logs + check monitoring + restart' are an anti-pattern.

On the execution side, a Zod schema defines the output contract, the prompt supplies only boundaries and context, analyzeIntent revalidates the model's output, and routeIntent dispatches strictly by mode. Missing slots are asked one at a time and pending slot-fill state expires. Above mode sits a gate: missing slots lead to clarify, high risk leads to a human-in-the-loop confirmation, and only extremely low confidence triggers follow-up, with 0.35 as a suggested starting threshold. Evaluation guidance at the end calls for per-intent accuracy instead of overall accuracy, fixing only the first broken layer when misclassification happens, and letting new intents represent new goals rather than new wording.

Takeaways

— A production intent result is a four-field object: intent, mode, slots, and missingSlots.

— Keyword-based intent detection fails on negation, anaphora, ellipsis, and synonyms, so `includes` checks belong nowhere near the main classifier.

— The model chooses the intent and fills parameters; engineering code validates format, whitelist membership, permissions, risk, and execution.

— Hardcoded rules still cover high-risk confirmation, sensitive-data masking, per-tenant isolation, and tool-name whitelists.

— Each intent definition needs a description, positive examples, negative examples, required slots, and allowed modes.

— Intent and tool should stay decoupled: one intent can span multiple tool steps, one tool can serve multiple intents, and compound intent labels for every workflow are an anti-pattern.

— A Zod schema defines the output contract; prefer the provider's native structured output or function calling, then revalidate with safeParse.

— analyzeIntent rejects unknown intent ids, modes outside the intent's allowed list, and schema-invalid output rather than routing a half-correct result.

— routeIntent reads only the validated mode: answer, clarify, execute, or reject; it never re-reads the user's raw message.

— When a slot is missing, ask only for that slot, and give pending slot-fill state an expiry so a delayed answer like 'production' cannot attach to an old question.

— Anaphora needs recent turns plus confirmed structured state, not the whole chat history; currentService updates only after tool success or explicit user selection.

— The gate order is missing slots, then high risk, then extremely low confidence; 0.35 is the proposed starting threshold rather than 0.8.

— Evaluation should split intent, mode, slot filling, unnecessary clarification, and wrongful execution; per-intent accuracy matters because overall accuracy hides class imbalance.

— When fixing a misclassification, find the first broken layer and fix only that layer; a new intent should represent a new goal, not a new way to phrase an old one.

Conclusions

Treating the model's structured output as untrusted input applies the same discipline used at any external boundary; the more an agent runs unattended, the more the validation contract matters.

The warning that confidence is observational, not a reliable probability, is a useful correction to agent demos that display a score as if the model had calibrated itself.

Updating currentService only after a tool succeeds or the user confirms prevents a subtle failure: a guessed service name from one failed lookup can poison all later anaphora.

Decoupling intent from tools is an architecture rule that prevents label explosion; a classifier meant to understand goals degrades into a workflow registry once every combination becomes its own intent.

The debugging advice to fix only the first broken layer trades an easy one-line patch for a structural fix at the right boundary; adding a keyword exception is how the original mess comes back.

Concepts & terms

Intent recognition

For conversational agents, the task of deciding what the user wants from an utterance and conversation context before choosing whether to answer, execute, clarify, or reject.

Slots

The concrete parameters an intent needs to execute, such as serviceId, environment, and timeRange; a slot is null when the input gives no evidence for it.

Mode

The disposition of an intent result: answer for explanation, execute for action, clarify for asking missing slots, reject for out-of-scope or forbidden requests.

Structured output

An LLM API mode that returns data conforming to a declared schema, usually JSON or function-calling arguments, so program code can validate and dispatch on it.

Function calling

A model API feature that lets the model select a declared function and fill its arguments as structured JSON instead of writing free text.

Zod

A TypeScript-first schema library used here to define the intent result contract and revalidate the model's response with safeParse.

HITL (human-in-the-loop)

A checkpoint where an agent interrupts before a risky action and waits for a person to confirm the proposed operation; confirmation is a gate, not a mode.

Source: juejin.cn ↗ Google Translate ↗ Backup ↗