A RAG Internship Interview That Tested Chunking, Hybrid Search, and Agent Architecture
The interview questions mirror what production AI engineering teams actually ask: not trivia about model names, but tradeoff reasoning about chunking strategies, hybrid search weighting, and cost-aware embedding choices. A candidate who can walk through why BM25 rescues vector recall from semantic drift, or why 2048-dimension embeddings are worth the storage cost, demonstrates the kind of systems thinking that separates demo builders from engineers who ship.
A student walked into an internship interview and got grilled for 90 minutes on the internals of a production RAG system called Paicongming. The questioning moved from multi-level semantic chunking without overlap and parent-document retrieval, through embedding model selection and cost analysis for 2048-dimension vectors, to the precise mechanics of a hybrid KNN+BM25 recall pipeline where BM25 dominates the final ranking to catch keyword-specific answers that vector search misses.
The interview then shifted to AI engineering practice: how 60% of the project's boilerplate was generated by Codex while core logic required manual tuning, a three-layer verification strategy for AI-generated code, and a crisp breakdown of Tool (atomic function), MCP (standardized tool protocol), and Skill (workflow encapsulation with prompt templates and scripts). The candidate also outlined a ReAct-loop upgrade path to turn the RAG system into an agent with a planner, tool calling, and human-in-the-loop safeguards.
Throughout, the interviewer pressed on tradeoffs: why skip overlap in Chinese text chunking, why let BM25 outweigh KNN, and why DeepSeek over Qwen for the LLM. The answers reveal a team that benchmarked alternatives, calculated storage costs, and learned that RAG effectiveness is 70% data quality, 20% retrieval design, and only 10% model choice.
Overlap in chunking is often treated as a default best practice, but this team's data shows it actively harms Chinese retrieval by polluting the topK with near-duplicates — a reminder that NLP defaults from English-centric research don't always transfer.
The decision to let BM25 dominate KNN in hybrid search is counter-trend: most systems treat vector search as primary and keyword search as auxiliary. Their A/B test showed the opposite produces better results for factoid questions where exact term matching matters more than semantic similarity.
The 70/20/10 split (data/retrieval/model) is a useful heuristic that pushes against the instinct to fix RAG problems by swapping models. Most gains come from chunking and retrieval design, which are less glamorous but higher-leverage.
An internship candidate who can discuss HNSW graph structures, BM25 term-frequency saturation, and ES dense_vector tradeoffs signals that Chinese CS programs are producing students with deeper infra knowledge than typical new-grad interview loops test for in Western markets.
The Skill description-as-classifier problem is under-discussed: a poorly scoped description field silently breaks agent behavior by either failing to trigger or triggering spuriously, making it a de facto configuration bug surface that most Skill authors don't test.
The parsing stage of a RAG pipeline draws practical questions: one person wrestling with LaTeX-to-Markdown conversion asks whether Apache Tika's output format suits semantic chunking. On the agent side, model selection emerges as a cost-accuracy tradeoff — Claude leads on accuracy for agent tasks while DeepSeek handles routine automation, and interviewers may probe that selection logic.
Lately I've been thinking about adding a RAG project. Just finished the parsing module and ran a benchmark — I wanted to parse LaTeX. Compared a hand-written parsing logic against LaTeXML's parsing in one test. Then realized I'd need to translate it back to Markdown for better semantic chunking, so I started working on that... Parsing really is kind of tricky hahahaha. What format does the output from Apache Tika 2.9.1 that the OP used turn into? Is it suitable for semantic chunking? [thinking]
Agent performance actually depends heavily on the underlying model choice. In my tests, Claude had the highest accuracy for Agent tasks but was expensive; for routine automation, DeepSeek was fully sufficient. If interviewing for an Agent role, the interviewer might also press you on model selection strategy.