AI Programming · Agent · Programmer

22 Interview Questions That Separate AI Engineers From AI Users

By Coffeeee · Jul 10, 2026

Read original on juejin.cn ↗ Google Translate ↗ Alt translation

The gap between prompting an LLM and engineering an Agent system is wide, and these 22 questions define the syllabus. A developer who can reason about chunk overlap, tool-loop recovery, and layered safety will ship reliable agents; one who cannot will ship demos that break in production.

Summary

A set of 22 interview questions maps the terrain from LLM internals—context windows, the Lost-in-the-Middle phenomenon, and sampling parameters—through Agent design patterns like ReAct and Plan-and-Execute, to the gritty details of tool definitions, error handling, and multi-agent coordination. Each answer supplies concrete tactics: why Temperature should sit near zero for tool selection, how to detect and break a Tool Loop, and what a layered defense against prompt injection looks like in code.

The second half shifts to production reality. RAG chunking strategies, context-window compression, and a three-tier evaluation framework get the same treatment as cost-control levers—token budgets, model tiering, and cache hits. A full Android technical-advisor Agent architecture and an on-device mobile Agent design close the loop, with explicit lists of which tools to expose and which to lock down.

Throughout, the emphasis stays on engineering trade-offs: when to split into multiple agents versus keeping one, why tool count above 20 tanks selection accuracy, and how to test a non-deterministic system with SLOs instead of pass/fail assertions.

Takeaways

— Lost-in-the-Middle causes accuracy to drop from 70–80% at the ends of a long context to 40–50% in the middle; front-loading key instructions and summarizing mid-context are the primary fixes.

— Temperature should be set to 0–0.1 for tool calling to keep tool selection stable; creative writing can push it to 0.7–0.9.

— Agent hallucinations are more dangerous than text hallucinations because they can fabricate tool calls, parameters, or even believe an action was executed when it wasn't.

— A ReAct loop needs a hard max_iterations cap to prevent infinite tool-calling cycles that can burn millions of tokens.

— Plan-and-Execute suits long, structured tasks like multi-file refactors; ReAct suits dynamic, step-dependent tasks like debugging or booking flows.

— Splitting into multiple agents is warranted only when roles conflict, knowledge domains are isolated, or observability demands it; otherwise a single agent with a good prompt is cheaper and faster.

— Tool definitions must read like API documentation: verbose descriptions, full semantic parameter names, strict JSON Schema types, and explicit exclusion scenarios.

— Tool error messages are new Observations for the LLM; a well-designed agent analyzes the error and retries with corrected parameters, not just reports failure.

— Registering 30 tools drops Tool Calling accuracy from ~95% to 75–80%; grouping tools into sets of 5–8 per level restores accuracy above 90%.

— System Prompts work better when short and specific; negative prohibitions can backfire by drawing attention to the forbidden behavior, and rules placed at the end are often ignored.

— Prompt injection defense requires layered measures: delimiters in the prompt, input escaping, parameter re-validation at the tool-execution layer, and pre-approval checks for destructive operations.

— RAG chunking needs 10–20% overlap to avoid cutting key facts at chunk boundaries; code and natural-language documents demand different chunking strategies.

— Context-window compression should be incremental and layered: keep the latest turns verbatim, summarize the middle, and extract structured facts from older turns.

— Agent evaluation is statistical, not binary; set SLOs like Tool Selection accuracy ≥ 90% and run regression tests after every model or prompt change.

— Production Agent systems require full-link tracing as the first infrastructure investment; without it, debugging is guesswork.

— Cost control is a design-time concern: per-conversation token caps, model tiering, cache hits, and prompt slimming prevent 100K-token single conversations.

— Database write operations from an Agent must be blocked at the tool level with SQL parsing that rejects DELETE/DROP/TRUNCATE, plus a secondary human-confirmation step.

— An Android technical-advisor Agent should deliberately not expose modify_code, delete_file, or deploy tools; generated code snippets must carry risk and version comments.

— On-device mobile Agents should handle intent classification and simple tool routing locally, offload complex reasoning to the cloud, and never let contacts or SMS leave the device.

— Testing non-deterministic Agent behavior requires running each test case multiple times and measuring the distribution of outcomes against an SLO, not a single pass/fail.

Conclusions

Tool-calling accuracy degrades exponentially with tool count, not linearly—a fact that makes tool grouping a hard requirement, not an optimization, once you pass roughly 20 tools.

The advice to keep System Prompts short collides with the equally common advice to write detailed tool descriptions; the real skill is allocating the token budget ruthlessly, cutting anything that doesn't directly improve tool selection or output quality.

Prompt injection is a fundamentally different threat in Agent systems than in chatbots because the attacker's payload can cross the text-to-action boundary; every tool-execution layer must treat LLM output as untrusted input.

Agent evaluation cannot borrow the pass/fail mindset of unit testing; teams that don't adopt statistical SLOs and regression suites will ship agents whose behavior drifts silently after every model update.

The ReAct vs. Plan-and-Execute debate is mostly a false choice in production—the hybrid pattern of a high-level plan with ReAct-style execution steps is what actually ships.

Cost control in Agent systems is not an ops concern but a design constraint; a single unguarded conversation can burn more budget than a month of normal traffic, so caps must be architectural, not reactive.

Concepts & terms

Lost-in-the-Middle

A phenomenon where LLMs pay disproportionately less attention to information in the middle of a long context window, with accuracy dropping from ~75% at the ends to ~45% in the middle, caused by primacy and recency biases in attention distribution.

ReAct (Reasoning + Acting)

An Agent pattern that interleaves reasoning steps with tool-calling actions in a loop: the model thinks, calls a tool, observes the result, and decides the next step, continuing until a goal is reached or a maximum iteration limit is hit.

Plan-and-Execute

An Agent pattern where the model first generates a complete plan of steps, then executes each step sequentially. It suits structured, long-horizon tasks but is less flexible when intermediate discoveries require changing direction.

Tool Loop (Agent Dead Loop)

A failure mode where an Agent repeatedly calls the same tool with semantically identical parameters, or two tools call each other in a cycle, consuming tokens without making progress. Detection uses pattern matching on call history; recovery uses prompt intervention, context truncation, or forced summarization.

Prompt Injection

An attack where a user's input contains instructions designed to override or bypass the System Prompt, potentially causing the Agent to execute unauthorized tool calls. Defense requires layered measures: prompt-level delimiters, input escaping, and re-validation at the tool-execution layer.

RAG Chunking Strategy

The method of splitting documents into retrievable pieces for a RAG system. Key parameters include chunk size (256–512 tokens), overlap (10–20%), and strategy (fixed-size, semantic, or recursive). Code and prose require different approaches.

Agent Eval SLO

A Service Level Objective for non-deterministic Agent behavior, such as 'Tool Selection accuracy ≥ 90%', measured statistically across multiple runs rather than as a binary pass/fail. Used to detect regressions after model or prompt changes.

From the discussion

The discussion is light, with no substantive engagement with the article's technical content. One exchange questions whether the author has moved beyond the article's scope to on-device models, while the other simply praises the article and wonders at the lack of likes.

See top comments, translated →

Source: juejin.cn ↗ Google Translate ↗ Backup ↗