跪拜 Guibai
← Back to the summary

Data Engineering Interviews Now Ask Which Warehouse Layer Feeds RAG

During an interview debrief, a candidate who had interviewed for a data development position at a certain company was suddenly asked by the interviewer in the middle of the second round:

❝"In the data warehouse you've worked on, which layer is most suitable for RAG retrieval? Why?"

He was stumped. He had memorized SQL rote questions, reviewed dimensional modeling, and prepared thoroughly for Spark tuning. Yet this particular question was neither a traditional data engineering question nor a pure large model question.

After the interview, he complained to me: "It feels like over the past two years, the question bank for data development roles has quietly changed. The exam questions have changed, but I didn't even get the memo."

This statement is very real. It's not that many data engineers suddenly lack ability, but that the evaluation coordinates of the role have shifted. Next, let's break this down clearly: why the questions changed, where they've shifted to, and how to prepare now.

1. It's not that the question bank updated, but the endpoint of the role changed

Comparing data development interview questions from two or three years ago with today's, the most obvious change isn't a few extra AI terms in the bank, but the ultimate consumer of the data has changed.

In the past, the main task of data engineering was "to prepare data well, for people to see." Data went through collection, cleaning, modeling, and aggregation, finally entering BI reports, business dashboards, and analysis systems to help business personnel make judgments.

Therefore, the traditional capability stack was always very stable:

These capabilities haven't expired. It's just that today, more and more data also needs to "serve AI." Agents need stable invocation, RAG needs accurate recall, and models need continuous improvement from real feedback. Thus, another layer has been stacked on top of the capability stack:

In one sentence

**❝**The previous endpoint was "BI for people to see." Now there's another endpoint: making data into context that AI can understand, retrieve, invoke, and trace. It's not that old questions are replaced by new ones, but that new usage scenarios have been added on top of old capabilities.

2. Don't rush to "learn another AI course"

Seeing the questions change, many people's first reaction is: quickly cram Transformer architecture, memorize Prompt Engineering, and review large model terminology.

Of course, you need to understand the basic concepts, but if preparation stops at the memorization level, the payoff is often limited. The reason is straightforward: standard answers can be found with a quick search; without real pipeline experience, you'll be exposed after two layers of follow-up questions; and more importantly, your most valuable data experience isn't being utilized.

**❝**A more effective route is not to retrain yourself into half an algorithm engineer, but to re-read the data engineering you've already done from an AI perspective.

Many seemingly unfamiliar new terms can actually find correspondences in old capabilities:

You can even take past projects and question them one by one:

Once this conversion is done, your positioning is no longer "a data engineer who knows a bit of AI," but "a data engineer who can stabilize AI data pipelines." The two sound only a few words apart, but the actual scarcity is completely different.

3 actions you can take immediately

**❝**Action 1: Rewrite old projects

For each data warehouse project you've done, add a section: if you were to redo it today with added RAG or Agent usage scenarios, how would the data model, permissions, quality metrics, and update mechanisms change?

**❝**Action 2: Run an end-to-end project

Don't just build a toy demo for show. The project must have real data, quantifiable metrics, exception handling, incremental updates, and traceable pipelines.

**❝**Action 3: Do a business solution drill

Assume the business side wants to launch an Agent. Provide a complete solution from the data side: where the data comes from, how to clean it, how to retrieve it, how to control permissions, how to monitor, and how bad cases flow back.

3. The 3 capabilities that truly create a gap

The first layer of questions tests whether you know something; the second and third layers will press whether you've actually done it. The following three capabilities are precisely where new questions are most likely to dig deep.

**❝**Capability 1: Connecting "Data → Embedding → Vector Database"

The key is not just connecting an Embedding API, but turning the entire pipeline into a data system that is deployable, scalable, and maintainable. You should at least be able to explain clearly:

You'll find that batch processing, performance tuning, incremental updates, and dual-version switching are originally the advantage zones of data engineering.

**❝**Capability 2: Understanding the boundaries of SQL, Graphs, Full-text, and Vectors

SQL excels at precise filtering and aggregation but is poor at complex relational reasoning; vector retrieval can find semantically similar content but is unreliable for precise conditions like product codes, amounts, and dates; full-text search is good for keyword matching but may not understand the user's true intent when they phrase things differently.

Production-grade RAG often doesn't "just pick one," but combines structured queries, graph relationships, full-text, and vector recall, then fuses the results through Rerank. Being able to clearly articulate the boundaries of each solution is more important than just reciting technical terms.

**❝**Capability 3: Truly running the data feedback loop

AI uses data, generating invocation logs; the logs expose failed requests and low-quality answers; the team then identifies bad cases from these and feeds them back into retrieval data, evaluation sets, or training sets.

This isn't just a phrase like "continuous optimization," but a data pipeline requiring definitions, versions, labeling, scheduling, quality monitoring, and effect evaluation. Model engineers focus more on the model itself; the data engineer's greatest advantage is precisely stabilizing the data flow within this closed loop.

4. A practical route more suitable for data engineers

If you want to truly develop the above capabilities, you can follow the four projects below step by step. They are not four isolated demos, but a complete pipeline from data warehouse governance to Data Agent.

**❝**Part 1: Financial Data Warehouse + Data Governance

Cover ODS, DWD, ADS layering, supplement master data, data contracts, and governance mechanisms like SLA/SLO. First, stabilize the trusted data foundation.

**❝**Part 2: Real-time and Offline Pipeline + Knowledge Graph

Build a real-time pipeline through Flink CDC, handle dual-stream synchronization, entity management, and graph relationships, so data can be queried not just by fields, but also by relationships.

**❝**Part 3: Search Engine + Semantic Retrieval

Combine Elasticsearch, BGE, hybrid recall, and Rerank to establish a quantifiable retrieval evaluation system, continuously optimizing Recall, Precision, and final answer quality.

**❝**Part 4: Data Agent + Feedback Loop

Add SQL sandboxes, Function Calls, bad case mining, and dataset feedback, stringing retrieval, invocation, security, and iteration into a closed loop.

Finally

Data engineering won't vanish into thin air, but it will continue to evolve. Similarly, frontend and backend won't disappear, but will become AI+, as will data analysis, etc. We must see the essence through the phenomena, the essence of the AI+ era.

This change isn't simply about replacing data development with AI, but requiring data development to further become "the data engineer for AI." You don't necessarily have to train models yourself, but you need to understand how AI consumes data, where it's prone to distortion, and how to make this pipeline stable, controllable, and traceable.