Turning a Million-Word EPUB into a Retrievable Knowledge Base with Milvus and Qwen
Most RAG tutorials stop at short documents. A real novel forces developers to confront chapter-aware loading, overlapping splits that preserve narrative context, and metadata schemas that make answers traceable — the same problems that surface with internal wikis, legal contracts, or any long-form technical documentation.
A complete RAG pipeline for a long EPUB starts with LangChain's EPubLoader, which preserves chapter boundaries instead of dumping the whole book into a single string. Each chapter then passes through a RecursiveCharacterTextSplitter configured with a 500-character chunk size and 50-character overlap, keeping martial-arts descriptions and character names from being severed at chunk boundaries. The chunks are embedded into 1024-dimensional vectors and written into a Milvus collection that tracks book ID, chapter number, and chunk index alongside the vector and raw text.
Querying flips the flow: a natural-language question gets embedded with the same model, Milvus runs a cosine-similarity search across the IVF_FLAT index, and the top three fragments are assembled into a prompt that tells Qwen to answer only from the provided excerpts. The system never re-reads the EPUB at query time, and the prompt explicitly instructs the model to admit when the fragments lack an answer.
The architecture cleanly separates concerns: the loader handles file I/O, the splitter manages chunk boundaries, Milvus owns storage and ANN search, and Qwen generates the final answer. The demo uses a single novel, but the schema already supports multi-book filtering through the book_id field, and the chunking parameters are presented as a starting point to tune against recall and latency.
Chunking a novel is a semantic problem, not just a mechanical one: too small loses narrative coherence, too large dilutes retrieval precision, and the right overlap preserves context that would otherwise be severed at arbitrary cut points.
The separation of concerns in this pipeline is unusually clean — each component (loader, splitter, vector store, LLM) has a single responsibility, which makes the system easier to debug and swap out than monolithic RAG implementations.
Storing chapter metadata alongside vectors is what turns a black-box similarity search into a system that can cite sources, a requirement that becomes non-negotiable once you move beyond toy demos.
The explicit instruction for the model to say 'I don't know' when fragments lack information is a small prompt-engineering detail that prevents the most common RAG failure mode: confident, fabricated answers.