AIGC

A RAG System Reads 22 RAG Papers to Explain RAG, for Under a Dollar

By 倔强的石头_ · Jul 27, 2026

Read original on juejin.cn ↗ Google Translate ↗ Alt translation

A working RAG pipeline that costs pocket change to run removes the last excuse for not building a private-document Q&A system. The combination of a free local embedder and a cheap cloud LLM means a team can stand up a cited, cross-document research assistant in an afternoon without writing code or committing to a subscription.

Summary

Twenty-two foundational RAG papers were fed into a local RAG system to create a research assistant that answers questions across the entire corpus. The setup pairs the open-source AnythingLLM desktop app—which handles PDF chunking, local vector embedding with multilingual-e5-small, and a LanceDB vector store—with a pay-per-use cloud model, deepseek-v4-flash, accessed via an OpenAI-compatible API. Embedding runs locally at zero cost; only the final chat inference touches the cloud bill.

The resulting assistant correctly distinguishes RAG-Sequence from RAG-Token in the Lewis 2020 paper, explains the Lost in the Middle phenomenon with its experimental design, contrasts Self-RAG’s introspection approach against CRAG’s corrective retrieval, and lists every paper in the set that proposes a new retriever or embedding method. A knowledge-transfer question on building a financial-document Q&A system yields six actionable recommendations grounded in specific papers. A negative-control question about diffusion models returns an honest “not found” instead of a hallucination.

Total cloud spend for ingesting 22 papers and running dozens of chat rounds came in under one yuan. The same pattern—swap the corpus, keep the toolchain—applies to internal API docs, policy libraries, or contract archives.

Takeaways

— AnythingLLM provides a desktop RAG app with no Docker dependency; PDFs are dragged in, chunked, and vectorized through a GUI.

— Using the built-in multilingual-e5-small embedder keeps vectorization local and free, while the cloud model is only billed for chat inference.

— deepseek-v4-flash, accessed via an OpenAI-compatible endpoint, handled English-to-Chinese cross-lingual Q&A across 22 papers for under 1 RMB total.

— Batch-uploading 22 PDFs at once caused failures; re-dragging the failed files without changing any settings succeeded on the second attempt.

— Six test questions confirmed precise single-document fact retrieval, concept explanation, cross-paper comparison, full-corpus summarization, practical knowledge transfer, and refusal to hallucinate on out-of-scope queries.

— The same zero-code stack generalizes immediately to internal API docs, legal policies, or insurance contracts by swapping the ingested PDFs.

Conclusions

Local embedding plus cloud chat is an under-appreciated cost pattern: the expensive GPU work (vectorization) stays on the user’s machine, and only the cheap token generation hits the meter.

Cross-lingual retrieval—Chinese questions over English papers—worked because the embedder maps both languages into a shared vector space, not because the LLM is multilingual. The retrieval step is the real multilingual bottleneck.

AnythingLLM’s batch-upload flakiness suggests the desktop RAG tooling layer still has rough edges that remind us these are early-stage products, not polished SaaS.

The negative-control test (asking about diffusion models) is the most important one in the whole demo; a RAG system that stays silent on unknowns is worth more than one that confidently fabricates.

Concepts & terms

RAG (Retrieval-Augmented Generation)

A technique that first retrieves relevant passages from a document store and then feeds them into a large language model’s prompt so the model can generate answers grounded in those sources, often with citations.

Vector Embedding

The process of converting text into a numerical vector representation. In RAG, both documents and user queries are embedded so that a similarity search can find the most relevant passages.

Lost in the Middle

A documented phenomenon where language models pay less attention to information placed in the middle of a long context window, causing retrieval performance to degrade when key passages are not at the beginning or end.

Self-RAG

A RAG variant where the model is trained to decide on its own whether retrieval is necessary for a given query, rather than always retrieving or never retrieving.

CRAG (Corrective RAG)

A RAG variant that evaluates the quality of retrieved documents and corrects or rewrites them before generation, aiming to reduce the impact of irrelevant or low-quality retrieval results.

Source: juejin.cn ↗ Google Translate ↗ Backup ↗