A RAG Internship Interview That Tested Chunking, Hybrid Search, and Agent Architecture

Lao Wang wore a black T-shirt today, with a line of white text on the chest reading 'My code does compile.' His stubble was a bit scruffy, probably hadn't shaved for days.

On the table sat a cup of iced Americano, likely just made, but he hadn't had a chance to drink it.

'I have high expectations for you,' Lao Wang said bluntly. 'Don't be nervous.'

(Inner monologue: Bro has seen big waves; not scared at all, bring it on.)

But honestly, this interview was really intense, almost too much. 😄

Folks, take a look at these questions first. Really, not just anyone can handle this.

Buckle up, let's goooooo～～

content

01. In your RAG project, how did you design the chunking? And how did you handle content parsing and vectorization? Also, how did you do retrieval and recall?

I said: 'I'll break it down into four parts: chunking, parsing, vectorization, and recall.'

Lao Wang nodded and leaned back in his chair, seemingly ready to hear my long speech.

Chunking: Paicongming uses multi-level semantic chunking, splitting progressively by paragraph → sentence → word → character. The default chunk size is 512 bytes, without overlap.

Lao Wang interjected: 'Aren't you afraid of semantic fragmentation without overlap?'

I said: 'We thought about that too. Later we found that overlap doesn't work well in Chinese scenarios; instead, it makes the same content appear twice in ES. During recall, the top results are all different slices of the same paragraph, essentially wasting topK slots. Our approach is to maintain a 1MB parent block outside the chunks, streamed in to prevent OOM. Chunks only handle vector recall; once hit, we trace back to the parent block to provide context to the large model.'

Lao Wang's eyes lit up. He didn't speak, signaling me to continue.

Parsing: We use Apache Tika 2.9.1, which auto-detects formats and supports PDF, Word, Excel, PPT, Markdown, and HTML. For PDFs, we handle them separately with PDFBox, slicing by page number while stripping out repetitive characters like headers and footers. For Chinese long sentence tokenization, we use HanLP's StandardTokenizer.

Vectorization: We use Alibaba Qwen's text-embedding-v4, dimension 2048, called via an API compatible with the OpenAI protocol, with a maximum batch size of 10 at a time.

Recall: This is the main event. We use ElasticSearch, with the index named knowledge_base. The vector field uses dense_vector, with cosine similarity. We implemented a hybrid recall of KNN + BM25. KNN first does a coarse recall of topK × 30 items, then BM25 re-ranks within this window, finally returning the topK.

After listening, Lao Wang said: 'Okay, you explained the whole thing clearly. Not bad.'

Though simple, this sentence gave me a lot of encouragement, honestly.

02. Why choose Qwen, and why choose the 2048-dimension vector?

I said: 'I actually researched this question.'

The Qwen v4 embedding model performs very well in Chinese scenarios, especially for technical documents with mixed content like code, English, and Chinese explanations.

We compared before: for the same Spring Boot tutorial, Qwen's recall accuracy was over ten percentage points higher than another vector model.

Lao Wang pressed: 'What about the 2048 dimensions? Doesn't Qwen v4 support three tiers: 1024, 1536, and 2048? Why choose the largest?'

Mainly two reasons.

First, accuracy. The higher the dimension, the stronger the vector space's expressive power, and the better the differentiation for long texts and technical terms.

Second, we calculated the storage and computation costs on the ES side. Compared to 1024 dimensions, 2048 adds about 8KB per vector. The knowledge_base index currently has hundreds of thousands of entries; the extra storage pressure is acceptable.

For retrieval, since BM25 handles the final re-ranking, the increased KNN latency is within an acceptable range.

Lao Wang tapped his finger lightly on the table: 'Good that you calculated the cost. Continue.'

Inner monologue: Heh, I got stuck on this question before, so I went back and did my homework.

03. How exactly did you implement this KNN vector recall and BM25 re-ranking?

KNN uses the native _knn_search interface of ES 8.x.

For specific parameters, we set recallK = topK × 30. For example, if we want to return 10 results, KNN first does a coarse recall of 300 items.

The BM25 part uses ES's rescore mechanism, re-ranking within the 300-item window recalled by KNN.

The weights are set to KNN 0.2 and BM25 1.0, with BM25 being dominant.

Lao Wang raised an eyebrow: 'Why is BM25 dominant and not KNN? Isn't the mainstream approach vector recall first?'

We ran A/B tests. Pure KNN in long-document scenarios can lead to 'semantically similar but answer-misaligned' situations. For example, if a user asks 'What is Paicongming's chunk size?', pure KNN will recall a bunch of paragraphs about chunking strategy, but the specific number '512' won't rank at the top.

BM25 is very sensitive to keyword hits and can push paragraphs containing precise terms like '512' or 'chunk-size' to the top. So we let BM25 make the final judgment, with KNN providing the candidate pool.

Lao Wang looked pleased, seemingly very satisfied with this answer.

04. What is BM25?

BM25 is a classic text relevance scoring algorithm, full name Best Matching 25, in the same lineage as TF-IDF.

Its core idea has three points: Term Frequency (TF), Inverse Document Frequency (IDF), and document length normalization.

The more times a term appears in the current document, the more important it is to that document; the more common a term is across all documents (like 'the', 'is'), the lower its weight; the longer the document, the appropriately lower the weight of a single term, as long documents naturally have an advantage.

ES's default similarity is BM25, with two parameters: k1 controls the term frequency saturation speed, default 1.2; b controls the length normalization strength, default 0.75. We didn't change these two parameters; we use the defaults.

Lao Wang pressed: 'What makes BM25 better than TF-IDF?'

The key is that the k1 parameter introduces term frequency saturation.

In TF-IDF, term frequency grows linearly; a document where a term appears 100 times scores 10 times higher than one where it appears 10 times. This is clearly unreasonable.

BM25 uses a (k1+1)*tf / (k1+tf) formula to cap the term frequency growth, which aligns better with human intuition.

05. What insights or feelings did you have while actually completing this project? How many people completed it?

The core development team for this project was just 3 people: me, two roommates, plus our mentor, Second Brother.

Speaking of insights, the deepest one is: For RAG, effectiveness is 70% data, 20% recall, and 10% model.

We initially obsessed over the model and tweaked prompts, but the results just wouldn't improve. Later, we went back to look at chunking and found that reducing chunk size from 1024 to 512, plus adding semantic boundary splitting, directly boosted recall accuracy by a large margin.

Lao Wang nodded: 'I agree with that feeling. Anything else?'

Another one: Production-grade RAG and demo-grade RAG are completely different things.

A demo can run the three-step process of vectorization + recall + LLM in a week, but adding permission isolation, multi-tenancy, file deduplication, streaming uploads, and incremental updates multiplies the workload by 10.

For permissions alone, we implemented three layers of filtering: userId, orgTag, and isPublic.

06. What role do you think AI played throughout this entire project?

For code generation, about 60% of the code was written by Codex, including mappers, service CRUD, ES mapping configurations, and Qwen API wrappers.

Lao Wang pressed: 'What about the remaining 40%?'

The remaining 40% is core business logic, like chunking strategy, hybrid recall weight tuning, prompt engineering, and performance optimization.

The first version AI writes for these is basically unusable directly; we need to repeatedly tune it based on actual data.

07. What is the response latency for ElasticSearch KNN vector recall?

We haven't done dedicated SLA testing, but from log instrumentation, the overall latency for a single KNN + BM25 rescore is roughly between 50ms and 120ms, averaging around 80ms.

Lao Wang pressed: 'What affects this latency the most?'

Mainly three factors.

First, data volume. As the data in the knowledge_base index grew from 100k to 500k entries, single queries became slower.

Second, the recallK setting. We tried topK × 50; recall quality didn't significantly improve, but latency increased, so we later settled on × 30.

Third, the document hit range. If the user's query keywords are too broad, the BM25 rescore window becomes larger, and latency increases.

08. Why did you use ES for this?

Mainly because a single index supports both text retrieval and vector retrieval.

If we used a specialized vector database like Milvus or Qdrant, and then separately set up a keyword search system, the operational and data synchronization costs would go up. ES 8.x's dense_vector is sufficient; there's no need to introduce new components.

Lao Wang pressed: 'Do you understand the underlying data structure of ES?'

For text retrieval, it's the classic Lucene inverted index. Documents are tokenized to build a mapping from terms to documents, and queries directly look up terms.

For vector retrieval, ES 8.x uses the HNSW (Hierarchical Navigable Small World) algorithm. Simply put, it's a multi-layer graph structure, entering from the top sparse graph layer and progressively refining the search downwards, finding approximate nearest neighbors in O(log n) complexity.

Lao Wang clearly had deep research in this area and nodded when he heard HNSW.

Regarding common query methods, ES mainly has several:

term exact match, match tokenized match, bool compound query, range query, and aggregation.

In our project, the most commonly used combination is bool + match + filter. Bool includes three usages: must, should, and filter. Should is used for permission filtering, and filter for hard filtering that doesn't participate in scoring.

Lao Wang looked pleased: 'Your understanding of this is solid.'

09. What AI models are you using?

We used two AI models in our project: one for embedding and one for LLM.

For embedding, we use Alibaba Qwen text-embedding-v4, 2048 dimensions.

For LLM, we use DeepSeek-chat, via the official API (api.deepseek.com/v1).

Lao Wang pressed: 'Why choose DeepSeek for the LLM instead of Qwen or GLM?'

Mainly cost-effectiveness.

DeepSeek's output quality is in the top tier, but the API price is only a fraction of GPT's, which is very friendly for us.

10. How do you usually use AI for programming?

Claude Code is mainly for processing text, and Codex is mainly for coding.

For convenience, I also integrated Codex into IntelliJ IDEA, which is very useful for reading code. I also built a PaiSwitch console for Claude Code to switch the underlying model; Kimi and GLM are both in use.

Lao Wang pressed: 'What Skills have you used with Claude Code?'

Frontend-design, Skill-creator, web-access, etc. The last one is the most useful; it can maximize the agent's internet connectivity. If some content requires login to access, this Skill can directly connect to Chrome's logged-in tabs via CDP.

11. How do you know if what the AI did is correct?

My current approach has three layers.

First layer, make the AI provide its own verification plan. For example, after it writes an API endpoint, I'll have it run unit tests again, or write a curl command to verify.

Second layer, critical paths must be reviewed by multiple Agents. For instance, after Codex writes code, I'll open Qoder's expert panel for testing.

Third layer, test it myself. Although Codex can use computer use to control the browser for testing, some stubborn problems are still unreachable by Agents right now and still require manual testing.

12. Do you usually follow cutting-edge tech trends? How do you follow them? In what way?

Mainly three channels: X, GitHub trending, and Hacker News.

On X, I follow a group of active developers and official accounts, including big companies like Anthropic, OpenAI, DeepMind, and some independent developers like swyx and karpathy.

For Hacker News, I run a scheduled task with OpenClaw that pushes an email to me every morning with the previous day's top 30 posts and comment summaries. I learn quite a bit from it.

13. If we wanted to upgrade this RAG into an agent, what would need to be done?

There are three core modifications.

• First, change retrieval from a 'one-time query' to an 'iterative query,' allowing the model to decide whether to query again and what to query based on intermediate results.
• Second, introduce tool calling. Besides knowledge base retrieval, add atomic capabilities like web search, SQL queries, and file reading/writing.
• Third, add a Planner, allowing the model to first break down the user's question into sub-tasks, then execute them one by one.

Lao Wang pressed: 'Specifically for the PaiCLI Agent, how would you modify it?'

The simplest solution is to introduce a ReAct loop.

Let the model think, then execute, then observe the result, then think again, repeating continuously.

Call ES to retrieve, observe ElasticSearch's return. Repeat until the model outputs a Final Answer to end the loop. This mechanism has ready-made implementations in LangChain and LangGraph; you can just move it over and use it.

However, to make it production-ready, a few more things need to be added:

Timeout and maximum round limits (to avoid infinite loops), intermediate state persistence (to avoid restarting everything on a single failure), and a human intervention interface (for confirming critical decisions).

14. How do you understand Skills, MCP, and Tools?

These three things often appear together in Claude Code, but their positioning is different.

Tool is the lowest-level atomic capability, like reading a file, running a command line, or querying a database. Each Tool is a function signature + a piece of execution logic. The model calls Tools through Function Calling, with parameters and return values being structured JSON.

MCP is a standardized protocol for Tools. MCP defines a unified protocol, allowing Tools to be packaged as independent Servers and called by any MCP-supporting client. Tools like Claude Code and Codex all support MCP.

Skill is the encapsulation of a workflow. It's a complete methodology for 'doing something,' containing prompt templates, call sequences, judgment logic, and even calls to sub-Skills. Anthropic places Skills in the .claude/skills/ directory; each Skill is a folder containing SKILL.md (instructions), references (reference materials), and scripts (helper scripts).

Lao Wang pressed: 'So are these three a containment relationship?'

Not exactly.

Tool is the atomic capability, MCP is the transport protocol for Tools, and Skill is a higher-order capability orchestrated from Tools and prompts.

You can analogize it like this: Tools are like commands in Linux like cat, grep, awk; MCP is like the shell, a unified interface; Skills are like written shell scripts.

Lao Wang couldn't help but laugh after hearing this analogy: 'That analogy is quite apt.'

15. In actual usage, during AI programming, which one do you think you use the most?

Skills are used the most, MCP is second, and native Tools are used the least.

Tools are low-level things; the Agent encapsulates them directly.

As for MCP, the ones I use most currently are Chrome Devtools MCP and IntelliJ IDEA, which can be integrated into the Agent for use.

For Skills, I use them quite a lot. If there are good open-source ones available on GitHub, I'll directly use them. If there isn't a suitable one, I'll have the Agent generate one based on my workflow.

Lao Wang pressed: 'Are there any pitfalls with Skills?'

The biggest pitfall is that if the Skill's description isn't written well, the model won't actively trigger it.

The description field of a Skill is essentially a classifier; the model decides whether to load this Skill after seeing the user's request. If the description is written too broadly, it will be falsely triggered.

16. Have you written Skills yourself? No? Have you looked at the specific implementation of Skills?

I've written quite a few.

For example, I previously distilled a Wang Xiaobo.skill, and the result was quite interesting.

Lao Wang's eyes immediately lit up: 'Can you talk about how you wrote it?'

The core of a Skill is three files.

The first is SKILL.md, with name and description in the frontmatter, and the workflow in the body. The second is the references directory, for reference materials like style guides and historical article samples. The third is the scripts directory, for helper scripts like a Python script to check word count.

After listening, Lao Wang let out a long breath: 'Okay, we're about done here.'

I asked back: 'Is it really necessary to go this hard for just an internship interview?'

Lao Wang smiled: 'Your understanding of RAG, Agent, MCP, and Skill is very solid, so the expectations are a bit higher. HR will talk to you shortly.'

When I walked out of the meeting room, I glanced at my phone. A full hour and a half.

Inner monologue: Lao Wang's stamina is really something else, he could still keep asking. I'm almost spent.