Open-Source Vector-Graph-RAG: Dual Engine of Vector + Graph to Break the Ceiling of Pure Vector RAG

I've been researching private deployment of enterprise AI knowledge bases, and I've been sharing high-value open-source AI projects.

In the past two years of researching private knowledge base solutions, my strongest feeling is: the ceiling of pure vector RAG is becoming increasingly obvious.

In early test scenarios, the requirements were simple: upload documents, do keyword search, provide basic Q&A. But as testing gets deeper, needs quickly get complex: 'Which cooperative suppliers are responsible for Zhang San's projects?', 'What solutions exist for similar equipment failures that occurred in Q3 last year?', 'Does this contract conflict with any clause in that policy?' These kinds of cross-document, relationship-finding, multi-hop reasoning questions almost always fail with pure vector RAG. It's not that the chunking isn't fine enough, or the embedding model isn't good enough; the underlying logic has limitations: pure vector retrieval is essentially 'fragment similarity matching'. It only recognizes whether text looks similar, and does not understand the relationships between entities. It's like someone who has only seen a pile of scattered notes, knows what each sentence says, but can't tell who is who or how things connect. When faced with questions that require stitching things together, they can only guess, and hallucinations naturally follow.

It was against this backdrop that Zilliz's open-source vector-graph-rag came to my attention. It doesn't overturn the existing vector RAG system; instead, it adds a knowledge graph recall layer on top of vector retrieval, using a dual engine of 'vector similarity + entity relationships' for hybrid recall, improving the accuracy and explainability of complex Q&A from the root.

This article will, from a technical researcher's perspective, fully break down the design philosophy, core architecture, technical highlights, integration cost, and selection boundaries, and explain clearly: what scenarios are worth introducing graph capabilities, and in which scenarios they are unnecessary.

The article is long; I suggest bookmarking it before you read.

1. Industry dilemma: four ceilings of pure vector RAG

Developers who have done enterprise knowledge base research generally agree: pure vector RAG works well for simple Q&A, but easily fails for complex Q&A. The deeper the testing, the more likely you are to hit four unavoidable problems.

1. Multi-hop reasoning fails, complex questions are answered inaccurately

Single-hop questions like 'Who is the supplier contact for product A?' can usually be answered correctly by pure vector RAG; but for multi-hop questions like 'How many delivery delays has the supplier corresponding to product A had in the past year, and what were the reasons?' which require cross-document, cross-entity, layer-by-layer association, accuracy falls off a cliff.

The reason is simple: vector retrieval is based on fragment similarity. It doesn't proactively expand along the relationship chain of 'product → supplier → delivery records → failure cause'. The information needed for the answer is scattered across different fragments in different documents; pure retrieval easily misses key fragments, and the final answer either lacks information or relies on the LLM to make things up.

2. Entities get misattributed, alias disambiguation is weak

In enterprise documents, multiple names for the same entity is the norm: a person has a full name, employee ID, nickname, and job title; a project has a full name, project code, and colloquial name. Pure vector RAG only matches by literal similarity, easily confusing different entities with similar names, or treating different names of the same entity as two completely unrelated objects.

We hit this pitfall in manufacturing-related tests: in the documents, 'Air Compressor Unit 1', 'No. 1 Air Compressor', and 'Air Compressor Station A Unit' referred to the same equipment. Pure vector retrieval treated them as three separate entities, completely fragmenting the information when answering fault questions, with extremely low accuracy.

3. Relationship Q&A relies on guessing, becoming a hotbed of hallucination

Questions involving 'relationships' are the most hallucination-prone scenarios for pure vector RAG. For example, 'What is the relationship between A and B?' or 'What cascading effects will this fault cause?' Vector retrieval can only recall fragments containing both keywords; the true logical relationship between them is entirely up to the LLM.

In many cases, the LLM fabricates relationships or misattributes entities just to make the answer sound fluent. In high-strictness scenarios like contracts, legal affairs, and operations, this kind of uncontrollable hallucination is fatal - users would rather have slower, less content than accept AI making things up.

4. Poor answer explainability, traceability only to fragments

Pure vector RAG's traceability usually only locates 'which fragment in which document'. But what users really want to know is: what entities and relationships is this conclusion based on? What is the information chain?

Fragment-level traceability doesn't solve the trust problem. Especially in government, enterprise, finance, and medical scenarios, every conclusion needs a clear evidence chain. If you only provide a few similar pieces of text, users won't dare use it directly and will still need to read the full document to verify. The knowledge base becomes an 'advanced search tool' and doesn't deliver much value.

These four problems can only be slightly improved by tuning prompts, changing embedding models, or optimizing chunking strategies; they can't be fundamentally solved. The root cause isn't retrieval technique, but that pure vector RAG from the start has no concept of 'entities and relationships'. To break through further, a new recall dimension is needed - that is, the knowledge graph.

2. Paradigm shift: from 'fragment similarity matching' to 'entity-relationship dual drive'

The core design philosophy of vector-graph-rag can be summed up in one sentence: Don't overturn the mature pure vector RAG system; instead, add a graph recall layer on top of it, forming a dual-engine architecture of 'vector retrieval ensures breadth, graph recall ensures precision.'

Here's a simple analogy: pure vector RAG is like a librarian who only searches by keywords. You ask a question, and he brings you a few seemingly relevant books; you have to find the answer yourself. Dual-engine RAG is like a senior researcher who understands both book classification and content associations - he not only finds relevant books, but also tells you the relationships between people, events, and organizations in the books, and follows the relationship chain to pull the relevant information together, finally giving you a well-supported conclusion.

Compared to the two modes, the core differences appear in four dimensions:

1. Depth of understanding: Pure vector only recognizes text fragments; dual-engine recognizes entities, relationships, and association paths. The former is 'looks similar'; the latter is 'logically related'.
2. Reasoning capability: Pure vector can only do single-fragment matching; dual-engine can do multi-hop expansion along graph relationships, supporting complex associative questions.
3. Entity consistency: Pure vector matches by literal text; multiple names for one thing cause information fragmentation; dual-engine does entity normalization and disambiguation, with all aliases pointing to the same entity, keeping information complete.
4. Explainability: Pure vector can only trace to text fragments; dual-engine can display the full entity-relationship chain, with every conclusion having a clear evidence path.

This is not about replacing vector retrieval, but complementing it. Vector retrieval is responsible for recalling semantically related long-tail content; graph recall is responsible for precise entity-relationship reasoning. The results are fused before generating the answer. This gives both the coverage breadth of pure vector RAG and the reasoning precision of graphs, making it a clear direction for next-generation enterprise RAG.

3. Five-layer dual-engine core architecture breakdown

The overall architecture of vector-graph-rag can be summarized as five layers: document preprocessing, entity-relationship extraction, dual-index storage, hybrid recall, and answer generation with validation. Each layer has a single responsibility, is backward-compatible with standard vector RAG, and extends graph capabilities upward. Existing vector knowledge base projects can be incrementally upgraded without tearing everything down.

Layer 1: Document preprocessing

Consistent with standard RAG flow, responsible for document parsing, format cleaning, and text chunking. It supports common formats like PDF, DOCX, TXT, and preserves document structure information to provide clean text material for later entity extraction.

The design philosophy here is to maximize compatibility with existing RAG flows. Whatever chunking and document handling you do now can essentially stay the same; you don't need to rebuild everything just because you added graphs. This is very friendly for upgrading existing projects, with low modification cost.

Layer 2: Entity and relationship extraction

This is the starting point of graph capability and the most core difference from pure vector RAG. The system automatically extracts entities, entity attributes, and relationships between entities from text fragments, and performs entity normalization - merging different names for the same entity and establishing a mapping system of standard names + aliases.

Extraction can combine rules with LLM: high-frequency, standardized entities are extracted with rules for efficiency; complex, long-tail relationships are extracted with the LLM for coverage. The extraction results retain the source fragment location information to provide evidence for later traceability.

Layer 3: Dual-index storage

Two sets of indexes are built simultaneously, each with its own role: Vector index is built on Milvus, storing vectors of text fragments for semantic similarity recall, exactly like traditional vector RAG; Graph index stores entity nodes, relationship edges, and corresponding attributes and source information. Both entities and relationships are bound to source IDs of original document fragments, ensuring every piece of graph data can find its original textual evidence.

There's a practical design here: the vector side continues to leverage Milvus's mature ecosystem, while the graph side is implemented lightweight, not forcing a heavy graph database. For small-to-medium projects, you don't need to deploy a separate graph database, lowering deployment and operation barriers.

Layer 4: Hybrid recall

This is the dispatch center of the entire solution. When a user question comes in, it doesn't just go down one retrieval path; it triggers two recall chains simultaneously: Vector chain embeds the question and recalls semantically similar text fragments from the vector library, ensuring information breadth; Graph chain first identifies core entities in the question, then expands along graph relationships, recalling associated entities, attributes, and relationship paths, ensuring logical precision.

The results from both paths are fused, deduplicated, and re-ranked to form the final context material. This avoids missing semantically related long-tail information (if only using graph) or missing key associated entities (if only using vector).

Layer 5: Answer generation and validation

The fused context material is submitted to the LLM to generate the final answer. The biggest difference from pure vector RAG is that the answer not only includes original fragment sources, but also displays the entity-relationship chain. Users can clearly see what relationships the conclusion was derived from and which documents/positions they came from.

An evidence validation mechanism is also added: if neither the graph nor the vector path has enough supporting evidence, the system explicitly states there is insufficient information rather than forcing the LLM to fabricate an answer, reducing hallucinations structurally.

4. Six-step working loop: from document ingestion to trustworthy Q&A

With the architecture understood, the full workflow is clear. From document upload to finally generating an answer with evidence, there are six stages forming a complete loop.

1. Document parsing and chunking

After upload, parse, clean, and chunk the document, retaining location info like structure, page numbers, paragraph positions. This is identical to traditional RAG, and existing logic can be reused directly.

2. Entity extraction and normalization

Perform entity-relationship extraction on chunked text, identify entities, attributes, and relationships; execute entity normalization: merge aliases, abbreviations, and different expressions of the same entity, create standard entity nodes, and point all aliases to the same node.

This step is the foundation for all subsequent graph capabilities. The more accurate the normalization, the higher the accuracy of later Q&A. The system supports human review and correction of extraction results; administrators can add aliases, fix wrong relationships, making it more accurate over time.

3. Dual-index construction

Text fragments are written to Milvus vector store; entities and relationships are written to graph index. Both are linked via document ID and fragment ID. Index construction supports incremental updates: when documents are added or modified, no full rebuild is needed; only the relevant parts are updated.

4. Question parsing and link expansion

When a user question enters the system, semantic understanding first identifies core entities and question intent. The vector side directly does semantic recall; the graph side starts from core entities and expands along relationship paths, recalling associated entities, attributes, and relationship edges, forming a complete evidence chain.

5. Dual-path recall fusion and re-ranking

Vector-recalled fragments and graph-recalled relationship data are uniformly deduplicated, sorted, and scored for credibility to select the most relevant material. The ranking considers multiple dimensions: semantic similarity, relationship strength, and source credibility, ensuring high-quality context.

6. Answer generation with evidence

The LLM generates the answer based on the fused material, and the output includes two types of traceability information: source text fragments and entity-relationship chains. Users can follow the chain to verify original evidence and ensure the answer is trustworthy.

If evidence is insufficient, the system explicitly says 'no sufficient evidence found' and does not force unreliable content. This is critical for enterprise scenarios - trust always matters more than answering a lot.

5. Five core technical highlights: why it beats pure vector RAG

There are many similar GraphRAG solutions; vector-graph-rag is worth attention because its design is very pragmatic, not chasing conceptual flashiness, but targeting real pain points in enterprise scenarios.

1. Entity normalization and disambiguation, solving 'multiple names for one object' from the root

This is a capability close to real enterprise scenarios. Many teams doing graph RAG only focus on extraction and skip normalization; the result is a pile of extracted same-name, alias entities, and information remains scattered, with limited improvement.

vector-graph-rag treats entity normalization as a core step, supporting alias mapping, rule merging, and human calibration. For the abbreviations, codes, and colloquial names that proliferate in enterprise scenarios, this step directly determines final Q&A quality. In our tests, just doing entity normalization well increased complex-question accuracy by more than 15%.

2. Controlled multi-hop reasoning, significantly improving complex Q&A

Pure vector RAG's multi-hop questions rely on the LLM making things up, while the dual-engine architecture actually expands along graph relationships. For questions like 'cascading effects, associative relationships, cross-entity statistics', accuracy improvement is obvious.

Moreover, the hop count is controllable: you can configure 1-hop, 2-hop, or multi-hop based on business scenarios. More hops give broader coverage but also add noise; the stricter the business, the more conservative the hop count. It's not about more hops being better; it's about configurability and control - that's the design an enterprise solution should have.

3. Dual-path recall complementarity, balancing breadth and precision

Many GraphRAG solutions tend to swing to the other extreme: over-relying on graphs and ignoring the breadth of vector retrieval, causing long-tail questions to go unanswered.

vector-graph-rag takes a balanced dual-path route: vector ensures breadth, covering long-tail, ambiguous, and semantically related questions; graph ensures precision, handling entity, relationship, and reasoning questions. After fusion, simple questions aren't degraded and complex questions improve significantly, with no obvious weaknesses.

4. Full-chain traceability, improving answer credibility

Unlike pure vector which can only trace to fragments, the dual-engine solution can show a complete evidence chain: which entities this conclusion comes from, through what relationships, and from which locations in which documents.

For high-strictness scenarios like contracts, legal, operations, and medical, explainability directly determines whether the content is usable. Only answers with a complete evidence chain are truly deployable in business.

5. Backward compatibility with vector systems, incremental upgrade

This is a very engineering-savvy design. It doesn't require you to abandon your existing vector RAG solution and build a graph system from scratch; you can incrementally add graph capabilities on top of an existing Milvus vector library.

For developers who already have a vector knowledge base, the cost of trying is low: no need to rebuild; just add extraction and graph layers to validate. If it doesn't work, you can roll back anytime without wasting a lot of work. This is ideal for iterative project upgrades.

6. Entrepreneurial perspective: implementation value and ROI estimation

From a technical researcher's perspective, I care more about whether this solution can stably improve Q&A quality and reduce rework. Based on our test experience, here are some evaluation references.

1. Quality improvement: higher project delivery satisfaction

Pure vector RAG can achieve 70%-80% accuracy in simple Q&A, but drops to 50%-60% on complex associative questions, easily producing biased answers. After introducing the dual-engine graph capability, accuracy on entity-relationship and multi-hop reasoning questions typically improves by 20%-30%, raising overall Q&A quality significantly.

2. Differentiated competitiveness: higher pricing

Current vector RAG implementations are numerous and quite homogenized. Solutions with graph + vector dual-engine capability have stronger complex reasoning in entity-relationship-heavy industry scenarios, especially manufacturing, legal, medical, and finance, where associative Q&A and traceability are more needed.

3. Development cost: save over 80% of effort compared to building in-house

If a team builds a graph RAG capability from scratch, they'd need to invest in modules like entity extraction, graph construction, hybrid recall, and traceability display, with long development cycles and a lot of tuning and troubleshooting. With open-source vector-graph-rag as the base, the core capabilities are already packaged; you only need to adapt to your business, do frontend display, and integrate with existing systems, allowing POC validation in a short time. For small teams, this is a low-cost way to validate graph RAG.

4. Operational cost: lightweight architecture, no new heavy components

Many GraphRAG solutions require a dedicated graph database (like Nebula, Neo4j), adding operational burden. vector-graph-rag can do lightweight graph storage on top of the existing Milvus ecosystem, so small-to-medium projects don't need to deploy a heavy graph database, adding little operational cost. Of course, for ultra-large entity volumes, you can connect a professional graph database if needed; scalability is sufficient. This design is thoughtful: light for small scenarios, upgradeable for large ones, fitting different scales.

7. Selection guide: which scenarios are worth it, which are not

Not every knowledge base project needs graph RAG. Tech selection is always about matching scenarios, not chasing the new. Based on our testing experience, here are clear applicable and non-applicable boundaries.

Recommended scenarios:

1. Manufacturing and operations knowledge base: Complex relationships among equipment, faults, spare parts, and processes; many multi-hop reasoning questions; graph improvement is most obvious.
2. Contract and legal knowledge base: Many entities, strict relationships, high trust requirements; evidence-chain traceability is of great value.
3. Enterprise HR/organization knowledge base: Complex relationships among people, departments, positions, and projects; many aliases; entity normalization solves many misattribution issues.
4. Medical/financial professional knowledge base: Many specialized terms, strict entity relationships, zero tolerance for hallucinations; dual-engine recall plus evidence validation is more controllable.
5. Medium-to-large enterprise knowledge bases: Large document volume, many entities, complex relationships; pure vector has hit its ceiling, and adding more documents won't improve Q&A quality.

Scenarios where it's not needed yet:

1. Small simple knowledge bases: Only dozens or hundreds of documents, mainly simple FAQ questions; pure vector RAG is fully sufficient; adding graphs is over-engineering.
2. Mostly unstructured content: Primarily images, audio/video, or pure long-text narrative, with few entities and relationships; graphs add little value.
3. Very limited project budget: Graphs require additional extraction, review, and operation costs; if budget and manpower are insufficient, it's better to solidly implement pure vector first.
4. Pursuing extreme Q&A speed: The graph path adds some retrieval latency; for extremely low-latency requirements, performance needs evaluation.

In short: when your project is stuck at the ceiling of pure vector RAG, with frequent inaccurate or untrustworthy answers, that's the best time to introduce graph capability. If pure vector already meets your needs, don't add complexity for the sake of tech flashiness.

8. Horizontal comparison: pros and cons of three mainstream approaches

There are several enhanced RAG routes on the market. We compared three mainstream approaches for your selection.

Type	Representative	Advantages	Limitations	Suitable for
Pure vector RAG	LangChain + Milvus / various RAG frameworks	Simple, mature, rich ecosystem, low cost	Weak on complex Q&A, entity confusion, hallucinations, poor explainability	Small projects, simple Q&A scenarios
Microsoft GraphRAG	microsoft/graphrag	Strong graph capability, high community interest	Heavy architecture, many dependencies, complex deployment, high hardware requirements	Large teams, ample operations and algorithm capability
vector-graph-rag	zilliztech/vector-graph-rag	Lightweight, compatible with Milvus ecosystem, dual-path recall, easy implementation, low upgrade cost	Graph depth slightly inferior to heavy solutions, ecosystem still evolving	Startup teams, enterprise delivery projects, incremental upgrade of existing vector libraries

Objectively, for ultra-large-scale projects with extremely high graph depth requirements, Microsoft GraphRAG has a higher capability ceiling; but for the vast majority of developers who already have a vector knowledge base and want a low-cost fast upgrade, vector-graph-rag is the more cost-effective choice.

Its greatest advantage is pragmatism: it doesn't pursue big-and-comprehensive, doesn't do complex concepts, but solidly implements graph capability on top of the mature Milvus vector ecosystem, letting teams get more stable Q&A results at minimal cost. This is what matters most in tech selection: don't choose the most advanced; choose what best fits your business stage.

9. Objectively viewing boundaries: it's not a silver bullet, these issues need attention

I think an evaluation should not only discuss advantages but also boundaries and limitations, to avoid overly high expectations and test pitfalls. Here are some experiences:

1. Extraction quality depends on the LLM's capability

The quality of entity and relationship extraction directly determines graph quality. If the underlying LLM is insufficient, extraction will have errors, wrong relationships, and final Q&A quality will suffer. For specialized domains, domain fine-tuning or supplementary rules are often needed to achieve ideal results.

2. Graph operation is long-term work

It's not a one-and-done process where you upload documents and the graph is automatically built. Maintaining entity aliases, correcting wrong relationships, and maintaining new entities all require ongoing work. The more specialized the domain, the greater the operational workload. To get good results, you can't skip manual review and calibration.

3. It adds some system complexity

Compared to pure vector RAG, there are extra stages: extraction, graph storage, and hybrid recall. Development and operations complexity will increase. Although it's already lightweight, it's still one layer more than pure vector. Small teams should evaluate their maintenance capacity.

4. Not all questions will improve

For simple factual Q&A and single-fragment answers, dual-engine and pure vector differ little. Improvements are mainly in complex relationships, multi-hop reasoning, and entity disambiguation. Don't expect all questions to become accurate after adding graphs; it solves specific pain points.

5. The ecosystem is still rapidly iterating

The project hasn't been open-source long; surrounding tools, best practices, and third-party integrations are still being improved. To have a plug-and-play complete product, some secondary development is needed. It's better suited for developers with technical capability to do customized validation.

Final thoughts

In these two years of researching knowledge base technology, my biggest takeaway is: pure vector RAG's popularization and education is complete, but its ceiling is increasingly clear. When users shift from 'can it answer' to 'is it accurate and trustworthy', optimizing chunking, changing embeddings, or tuning prompts won't bring fundamental improvement. The competition in next-generation RAG will be over multi-source recall capabilities - vector, graph, structured data, business system data - with multiple paths fusing and complementing each other.

vector-graph-rag represents a very pragmatic route in this direction: no overturning, no flashy concepts; incrementally adding graph capability on top of an existing mature vector system, helping break through the performance bottleneck at minimal cost.

Based on our extensive testing practice, AI knowledge base selection is never about picking the most complex, but about picking what best matches your business stage.

If you're troubled by the ceiling of pure vector RAG, I strongly suggest pulling down the test suite and running the test set to see the actual results. Sometimes a single layer of graph capability can open new technical ideas.

This is a personal technical evaluation. All conclusions are based on actual test experience in the project. Please complete a full POC verification and stress testing before production deployment.