Background

This article prepares some interview questions on AI Agents. It's more like notes than an article, meant for future review and consolidation of key knowledge points. Why do this? Because summer is here. Friends who know me well are aware that I've been an unemployed old man for two summers in a row. Although I'm still safe this year, who knows what will happen later? So I'm preparing in advance, so I don't go out and compete for jobs with others completely unprepared. How could I possibly win?

Speaking of interview questions, do you think understanding these theoretical things is useful? I think you still need to master some of them. Otherwise, if everyone can use AI to build things and generate code to improve efficiency, what's your competitive edge when you go for an interview? Are you more handsome than others, or do you have more tokens? On what basis will the interviewer choose you over someone else? I think this requires us to master more things that we might not normally think about. The goal is that one day, when you sit face-to-face with the interviewer and he asks you something, you won't just smile and say "I don't know." This article has eight parts and 22 questions. It's recommended to make a cup of coffee before reading, calm down, and take your time.

1. LLM Foundational Knowledge

Q1. Please explain what the LLM context window is, and how the Lost-in-the-Middle phenomenon occurs?

The context window is the maximum number of tokens an LLM can process at one time, determined by the attention mechanism in the Transformer architecture. Tokens within the window are visible to each other via Self-Attention; the model cannot perceive information outside the window.

The Lost-in-the-Middle phenomenon refers to: when the input context is very long, the model utilizes information at the beginning and end most effectively, while utilization of information in the middle part drops significantly. The model's accuracy is about 70-80% when information is at the beginning or end of a document, but can drop to 40-50% when it's in the middle.

Reasons: Attention distribution is uneven; the model tends to focus on early sequence content (primacy bias) and recent content (recency bias), and the middle part is easily diluted.

How to prevent it

1. Front-load/Back-load key information: Put the System Prompt at the very beginning, important instructions at the start, and the latest information at the end.
2. Structured Prompts: Use Markdown/XML tags to help the model perceive content boundaries.
3. Sliding Window + Summarization: Periodically summarize long conversations, compress the middle part, and place it at the beginning.
4. RAG Result Ordering: Re-rank retrieved documents by relevance, placing the most relevant ones at the beginning and end.
5. Adaptive Truncation: Prioritize discarding low-value historical conversations from the middle part.

Q2. What are the differences between Temperature, Top-P, and Top-K parameters, and how do you tune them in actual deployment?

Core Differences:

Parameter	Function	Value Meaning
Temperature	Controls the smoothness of the probability distribution	Lower = more deterministic (0 = greedy), higher = more random
Top-P (Nucleus Sampling)	The smallest set of tokens whose cumulative probability exceeds P	0.9 = take the set of token candidates with a cumulative probability of 90%
Top-K	Only keep the K highest-probability tokens	K=40 means only sample from the top 40 tokens by probability

Practical Tuning Suggestions:

Scenario	Temperature	Top-P	Top-K
Code Generation / Function Calling	0～0.2	0.9～1.0	Disable
Factual Q&A (RAG)	0～0.3	0.9	Disable
Creative Writing / Brainstorming	0.7～0.9	0.9～0.95	40～50
Customer Service Reply (Standardized)	0.1～0.3	0.85	Disable
Agent Tool Selection	0～0.1	Disable	Disable

Suggestions

• For the Agent's Tool Calling phase, it's recommended to set Temperature close to 0 to ensure stable tool selection and reduce hallucinated calls.
• During the reply generation phase, Temperature can be slightly increased to make expressions more natural.
• Do not use Top-P and Top-K simultaneously; Top-P is generally more versatile.
• Online, it's recommended to do A/B testing with a granularity of 0.05～0.1 for fine-tuning.

Q3. How does model hallucination occur? How can it be mitigated in Agent scenarios?

Roots of Hallucination:

1. Data Level: Errors/contradictory information in pre-training data is internalized by the model.
2. Architecture Level: The model is essentially a "next token predictor," not a factual database.
3. Alignment Level: RLHF encourages the model to be "helpful," sometimes suppressing "I don't know."
4. Knowledge Cutoff: For new knowledge after model training, it can only "guess."

Hallucination Risks in Agent Scenarios (much more severe than plain text):

• Tool Hallucination: The model calls a tool or parameter that doesn't actually exist.
• Parameter Hallucination: Passes a correct parameter name to a tool but with a non-existent value (e.g., passing a fake city code).
• Result Hallucination: The tool returns data, but the model misinterprets it when summarizing.
• Action Hallucination: The model believes it has called a tool, but actually hasn't (thinks it performed an action but didn't execute it).

Mitigation Methods

Method	Principle
Tool Calling + Structured Output	Make the model output structured JSON instead of free text.
Constrained Decoding (Grammar Guidance)	Constrain model output using JSON Schema / GBNF.
Retrieval-Augmented Generation	Provide the model with factual context to reduce the need to "fabricate."
Self-Consistency / Multi-path Verification	Infer multiple times and take the majority answer.
Citation Tracing	Require the model to cite information sources; discard those that cannot be traced.
Confidence Scoring	Have the model provide a confidence score; do not display low-scoring results.
Human-in-the-Loop	Key decisions (payments, deletions) are confirmed by a human.

2. Agent Architecture and Patterns

Q4. Hand-write the core loop of a ReAct Agent

The role of max_iterations

Prevents infinite loops and token explosion. An Agent can fall into a dead loop of continuously calling tools. Each loop consumes tokens; without an upper limit, a single conversation could consume millions of tokens.

How to implement parallel Tool Calling

If there are no dependencies between tools, you can have the LLM generate multiple tool_calls in a single inference step, then execute them concurrently. Similar to Promise.all / asyncio.gather.

Q5. What is the difference between ReAct and Plan-and-Execute? What scenarios are they suitable for?

Dimension	ReAct	Plan-and-Execute
Execution Method	Think and act step-by-step, progressive reasoning	First formulate a complete plan, then execute step-by-step
Flexibility	High, can change direction anytime	Low, plan is not easily changed after formulation
Explainability	Intermediate reasoning steps are visible	Plan + execution steps are clear
Long Task Performance	Easy to go off-track or into dead loops	Clear structure, suitable for long workflows
Planning Overhead	No explicit planning cost	Planning phase requires extra tokens
Error Recovery	Naturally supported (can adjust anytime)	Requires a re-planning mechanism

Scenario Selection

Scenario	Recommended Mode	Reason
Single-step Q&A / Simple Query	ReAct	No planning needed, answer after query
Multi-step Tool Use (e.g., booking tickets)	ReAct	Each step depends on the previous result, requiring dynamic decision-making
Complex Document Generation (e.g., weekly report)	Plan-and-Execute	Define structure first then fill in content, ensuring completeness
Code Modification / Multi-file Refactoring	Plan-and-Execute	Need to understand the whole before making changes
Unknown Exploration Tasks (e.g., research)	ReAct	Intermediate discoveries may change direction
Troubleshooting / Debug	ReAct	Gradually narrow scope, flexible adjustment

The most common approach in actual production is a hybrid of the two—generate a High-Level Plan at initialization, but use the ReAct pattern for fine-grained decision-making during the execution of each step, allowing re-planning when blocked. This is the Plan-and-Solve + ReAct hybrid pattern.

Q6. Have you designed a Multi-Agent system? Explain how you decide when to split into multiple Agents, and how Agents communicate with each other?

When should you split into multiple Agents?

1. Role Conflict: A single Agent needs to both strictly check and creatively answer, creating a role contradiction.
2. Knowledge Domain Isolation: Needs to access different data sources/tool sets; putting them together decreases Tool selection accuracy.
3. Different Reasoning Strategies Needed: One Agent needs rigorous step-by-step reasoning, another needs quick intuitive judgment.
4. Cross-System Boundaries: Needs to call resources in different permission domains.
5. Observability Requirements: Need to track the time consumption and success rate of each link separately.

Disadvantages of Multi-Agent

1. Multi-Agent increases latency, token cost, and coordination complexity.
2. The coordination cost of a dialogue system with more than 3 Agents rises exponentially.
3. If a problem can be solved by one Agent + a good Prompt, splitting it into multiple Agents is over-engineering.

Agent Communication Patterns

Suggestions

• Messages between Agents need a structured protocol (cannot be plain text dialogue; a message Schema must be defined).
• A timeout mechanism is needed (one Agent getting stuck cannot block the entire system).
• It's recommended to introduce shared context (a global blackboard / shared Memory) to reduce the token cost of repeatedly passing information.

Q7. What are the detection and recovery strategies for Agent dead loops (Tool Loop)?

Typical manifestations of a dead loop:

1. Repeatedly calling the same tool with different parameters that are semantically identical.
2. Tool A and Tool B calling each other, using each other's output as their own input.
3. Continuous error → retry → error → retry.
4. The reasoning phase constantly "self-doubts" without producing an action.

Detection Strategies

Recovery Strategies

Strategy	Action	Applicable Scenario
Prompt Intervention	Insert a system message prompting the Agent to change direction	Mild dead loop
Context Truncation	Discard redundant dialogue from the middle part	Agent has accumulated too much context
Forced Summarization	Have another LLM compress the current progress and restart the loop	Severe dead loop
Degradation Strategy	Fall back to a simple Q&A mode, stop tool calls	When unrecoverable
Human Handoff	Transfer to a human for processing	When all above fail

Engineering Safeguard Production environments must configure an absolute safeguard—after reaching the maximum number of turns or token budget limit, forcefully end the process and reply with "Currently unable to complete, please try simplifying your request" or transfer to a human.

3. Tool Calling and Function Calling

Q8. How do you define a "good" tool? What pitfalls have you encountered?

Typical bad tool definition

The reason it's bad: the description is too vague, parameter names are meaningless, and the model doesn't know when to use it.

Correct definition

Key Pitfall Experiences

Pitfall	Phenomenon	Solution
Description too short	Model doesn't know when to use this tool	Write clearly "when to call / when not to call"
Parameter name too short	q / id / val, model doesn't know what to fill in	Use fully semantic nouns
Missing type constraints	Passes string "123" but API expects a number	Strictly annotate JSON Schema types
Incomplete enum values	Model fabricates values not in the enum	enum field constraint
Missing default values	Model doesn't pass optional parameters, causing uncertainty	Provide reasonable default values
No boundary description	User asks a non-technical question, but the knowledge base tool is also called	Declare exclusion scenarios in the Description
Too many tools	The probability of the Agent selecting the wrong tool increases	Group/hierarchically expose tools when exceeding 20
Tool competition	Two tools have overlapping functions, the model repeatedly hesitates	Ensure tool responsibilities are mutually exclusive

Engineering Principle: The tool definition's Description is writing a manual for the model. Write clearly what this is, when to use it, when not to use it, and what the parameters mean. Treat tool definitions like API documentation; it's better to be verbose than vague.

Q9. What if a tool call fails? What is your error handling strategy?

Layered Error Handling Strategy

Code Implementation

Key Insight: Tool error messages are new "Observations" for the Agent. The LLM's behavior after receiving them determines whether the Agent is intelligent. A good Agent will analyze the error cause and retry with corrected parameters.

Q10. If an Agent is registered with 30 tools, what do you do if Tool Selection accuracy drops?

This is a very practical problem in Agent engineering—the difficulty for a model to choose correctly among too many tools rises exponentially.

Optimization Strategies

Tool Grouping

Intent-based Pre-routing

Dynamic Tool Registration

Tool Embedding Retrieval

Tool Aggregation (Composite Tool)

Encapsulate multiple fine-grained tools into one coarse-grained tool, performing secondary routing internally, for example:

Generally, when the number of tools increases from 5 to 20, Tool Calling accuracy drops from about 95% to about 75-80%. Grouping to 5-8 tools per level can restore it to 90%+.

4. Prompt Engineering

Q11. How do you design and iterate on a System Prompt? What counter-intuitive problems have you encountered?

System Prompt Design Methodology

Example Architecture

Counter-intuitive Pitfall Experiences

Pitfall & Counter-intuitive Point	Reason	Countermeasure
Shorter System Prompts work better	Long Prompts dilute key instructions	Delete all non-critical content; put important parts in the first 20%
Saying "Don't do X" sometimes triggers X	The model pays attention to the tokens of the negative description	Replace negative prohibitions with positive guidance
Rules placed at the end are often ignored	Lost-in-the-Middle phenomenon	Put the most important rules at the very beginning
Specific instructions are more effective than abstract principles	"Answer in no more than 3 points" is more effective than "be concise"	Use quantifiable constraints
Long Prompts significantly increase Token cost	System Prompt is carried in every conversation	Regularly evaluate "whether each rule is worth its tokens"

Q12. How do you design Prompts to prevent an Agent from being attacked by user injection (Prompt Injection)?

Prompt Injection is particularly dangerous in Agent scenarios because Agents have tool-calling capabilities. Attackers might induce the Agent to execute operations like deleting data or leaking information.

Defense Strategy (Layered Defense)

Specific Prompt-Level Implementation

Additional Engineering Measures:

• Perform special character escaping on user input (e.g., escaping <> to entities).
• Perform another round of parameter validation at the Agent's Tool Execution layer (do not trust any content from the LLM).
• For critical operations (payment, deletion, write operations), the Tool should call a Permission Check Tool for pre-approval before execution.

5. RAG and Memory Management

Q13. Have you designed a RAG system? Explain how you chose the Chunk strategy?

Chunking Strategy Decision Tree

Key Parameters

Parameter	Recommended Value	Explanation
Chunk Size	256～512 tokens	Too small: incomplete info; too large: too much retrieval noise
Overlap	10%～20% of chunk size	Prevents key info from being cut exactly at the boundary
Embedding Model	ada-002 / bge-large / text-embedding-3-small	For domain-specific data, consider fine-tuning or choosing a larger model
Top-K Retrieval	3～5	The info needed for an answer usually comes from 3-5 chunks

Pitfalls Encountered

1. No Overlap → Some key information is exactly truncated at the boundary of two Chunks and cannot be retrieved.
2. Chunk too large → One Chunk contains multiple topics; it's retrieved but the model doesn't know which part to use.
3. Only single-layer retrieval → Adding a Rerank layer can significantly improve the quality of Top-K.
4. Not differentiating document types with uniform parameters → Code and natural language documents should use completely different Chunk strategies.

Advanced

Q14. What do you do when an Agent's context window is full? What is your compression strategy?

A full context window is an unavoidable problem, especially for long-conversation Agents.

Layered Compression Strategy

Token Budget Allocation Strategy

Core Principle: The latest information is retained most completely, the middle part needs compression, key facts are structurally extracted. Don't wait until it's full to process; perform incremental compression after each dialogue turn.

6. Evaluation and Observability

Q15. How do you evaluate whether an Agent is good? How do you build an Eval system?

Agent Eval Three-Layer System

Eval Dataset Construction

Eval Workflow

Q16. Your Agent has a problem in the production environment; how do you troubleshoot it?

The difficulty of Agent troubleshooting lies in the fact that you cannot debug an Agent like you debug code, because the output can be different each time.

Troubleshooting Toolbox

One-Stop Troubleshooting Process

An Agent system without Tracing is like blind men touching an elephant. Before an Agent goes live, you must first integrate Tracing tools (LangSmith / LangFuse). This is the most important infrastructure investment.

7. Security and Guardrails

Q17. Have you designed security guardrails for an Agent? How do you prevent an Agent from doing things it shouldn't? What if an Agent calls a database DELETE?

Layered Security Model (Defense in Depth)

Coding Implementation Example

What if an Agent calls a database DELETE?

1. System Prompt explicitly prohibits "executing non-read-only SQL."
2. The database Tool internally performs SQL parsing; if DELETE / DROP / TRUNCATE is detected, it directly rejects it.
3. Write-operation Tools require secondary user confirmation + operation audit logs.

8. Engineering Implementation and Scenario Questions

Q18. Design an Android Technical Advisor Agent. How would you design the architecture?

System Architecture

Tool Definitions

Tool	Purpose	Permission
search_android_doc	Search official Android documentation	Read-only
search_tech_wiki	Search internal tech Wiki	Read-only
search_gitlab_code	Search GitLab repository code	Read-only
search_issue	Search Bugs / Requirements	Read-only
read_file_content	Read file content to understand context	Read-only
summarize	Summarize long text	No side effects
generate_code_snippet	Generate code snippets	Output text only

Tools deliberately not exposed:

• modify_code : Not exposed; Agent does not modify code.
• delete_file : Not exposed.
• deploy : Not exposed.

Boundary Handling:

• Non-technical questions: Intent classifier detects non-Android technical questions → politely refuse.
• Internal company information: Agent needs to confirm the user has permission to access the information; otherwise, say "No permission."
• Uncertain answers: Must say "I'm not sure about this" and provide a direction for searching.
• Code security: Generated code snippets must include comments explaining risks and applicable versions.

Cost Control:

• Simple questions use Claude Haiku / GPT-4o-mini; complex questions escalate to flagship models.
• Cache RAG retrieval results (direct cache hit for identical questions).
• Set a single conversation Token cap to prevent budget runaway.

Q19. You need to deploy an AI Agent online. What issues do you need to consider? Draw a deployment architecture.

Deployment Checklist

Latency

• Monitor P50 / P95 / P99 latency for LLM calls.
• Streaming output to reduce Time-To-First-Token.
• RB-Tree cache: return cached results directly for identical requests.
• Model degradation for simple questions (Claude Sonnet → Claude Haiku).

Cost

• Token usage monitoring + budget alerts.
• Caching strategies to reduce duplicate calls.
• Prompt compression to reduce tokens per call.
• Automatic summarization and compression for long conversations.

Reliability

• LLM API retry + fallback (main model down → backup model).
• Set a timeout for each Agent step → trigger fallback on timeout.
• Health Check: /ping → quickly detect if the Agent is healthy.
• Rate Limiting: prevent a single user from flooding the system.

Observability

• Full-link Tracing (time/tokens/tool calls per step).
• Business metric monitoring (success rate / user satisfaction / tool accuracy).
• Anomaly alerts (token spike / success rate plunge / latency spike).
• Log retention (the Agent's Thought process is the core basis for debugging).

Security

• User authentication (JWT / API Key).
• Tool call permission control.
• Input/Output security checks.
• Audit logs (all operations are traceable).

Q20. How do you control the call cost of an Agent? What if a user's single conversation costs 100,000 Tokens?

Cost control should be part of the system design, not something you check after a problem occurs.

Prevention

Measure	Effect
Per-conversation Token cap (e.g., 20K)	Prevents a single runaway
Max reasoning steps (e.g., 10 steps)	Prevents Tool Loop consumption
Model tiering (simple→small model, complex→large model)	Reduces cost by 50-70%
Cache hits (return cached results for identical RAG queries)	Reduces retrieval cost
Prompt slimming (remove non-critical instructions, use short names)	Saves 5-10% per turn
Tool grouping (only expose currently needed tools)	Reduces window space occupied by Tool Definitions

In-Process Monitoring

Post-Mortem Analysis

• Daily routine Token usage reports (categorized by user / conversation / Agent).
• Identify users with the highest CPU (Cost Per User)—is it normal use or abuse?
• Identify abnormal Token Spikes; a sudden surge usually means a bug.

Post-mortem investigation follows these questions

1. Did the Agent fall into a Tool Loop?
2. Did a tool return a massive amount of data?
3. Is the user's question something the LLM cannot handle?
4. Is the System Prompt or Tool Definition too large?

Then fix the root cause: add turn limits, truncate tool return results, slim down the Prompt.

Q21. If you were to implement a local AI assistant Agent on an Android phone now, how would you do it?

Overall Architecture

Trade-offs of On-Device Models

• Don't expect an on-device model to reason like GPT-4.
• On-device: Intent classification + Simple Q&A + Tool routing.
• Cloud: Complex reasoning + Long text generation.
• Critical data (contacts/SMS) never leaves the device.

Implementation Key Points:

1. Agent runs as a Foreground Service, resident in the background.
2. Use Notification quick replies as the interaction entry point.
3. Tool Calling is implemented via Android native APIs.
4. Use DataStore / Room for session persistence.
5. Requires Privacy Sandbox-level permission control.

Examples of On-Device Agent Tools:

Tool	Implementation	Permission
query_contacts	ContentResolver.query(ContactsContract)	READ_CONTACTS
read_calendar_events	ContentResolver.query(CalendarContract)	READ_CALENDAR
send_notification	NotificationManager.notify()	POST_NOTIFICATIONS
search_files	MediaStore or SAF	File access permission
take_screenshot	MediaProjection	Screen recording permission
open_app	PackageManager.getLaunchIntentForPackage()	No special permission
web_search	Network API call	INTERNET

Q22. In an Agent scenario, how do you test a non-deterministic system?

Core Insight: Testing an Agent is not a traditional "right or wrong" test, but a statistical one of "in what percentage of cases is the performance acceptable."

Layered Testing Strategy

Automated Evaluation Tools

Key Metrics

• Do not require 100% accuracy (impossible, because LLMs are probabilistic).
• Set SLO (Service Level Objective): e.g., Tool Selection accuracy ≥ 90%.
• Continuous monitoring: Regression test after every model upgrade / Prompt modification.
• Anomaly detection: If the success rate suddenly drops from 92% to 85%, it indicates a regression problem.

Conclusion

I think using AI now is a bit like driving a car. Almost everyone can drive now; putting it in gear and stepping on the gas isn't difficult. But if the car has a problem, like a flat tire, can you change it? If the car stalls halfway, do you know the reason? I even saw a video before where someone didn't know which hole to pour windshield washer fluid into. I don't know if it was real. AI is the same. Many people can use AI, but few understand AI. When everyone is using AI to write weekly reports, make PPTs, and draw pictures, if you can understand why the ReAct loop prevents dead loops and know how to optimize the Lost-in-the-Middle phenomenon, then you are no longer just a "user" of AI, but an "engineer" of AI.