Agent · LLM

Milvus from Prototype to Production: A Complete Walkthrough for RAG Memory

By 为你学会写情书 · Jul 14, 2026

Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Vector retrieval is the bottleneck in most RAG pipelines, and index choice alone can swing recall by 15 percentage points while multiplying memory costs. A wrong schema or forgotten `load` call breaks an Agent's memory silently in production.

Summary

A complete Milvus integration walkthrough uses an AI diary project to demonstrate every stage of building RAG memory. Schema design covers field types, dynamic versus strict schemas, and time-based partitioning for query performance. Index selection follows a decision tree from brute-force FLAT for small datasets through IVF_FLAT, HNSW, and quantized variants, up to DISKANN for billion-scale collections.

Search patterns include basic semantic retrieval, scalar-filtered vector search, range search with similarity thresholds, and multi-vector hybrid search with RRF re-ranking. Production concerns get equal weight: consistency levels from Strong to Eventually, explicit memory load and release, batch writes with flush, model selection trade-offs, and cost optimization through compression and TTL-based cleanup.

Takeaways

— FLAT brute-force search is 100% accurate but only viable below 100k vectors; IVF_FLAT handles millions at >95% recall and is the default starting point.

— HNSW delivers >98% recall and high QPS at the cost of substantial memory, making it the go-to for latency-sensitive production workloads.

— IVF_SQ8 compresses vectors to one-quarter size with only a modest accuracy drop, useful when memory is the binding constraint.

— DISKANN keeps indexes on SSD and targets billion-scale collections without requiring all data in RAM.

— Switching embedding models later forces a full re-index of every vector, so the model choice is effectively permanent once data is live.

— Collections must be explicitly loaded into QueryNode memory before any search; forgetting this is a common silent failure in production.

— Time-based partitioning combined with TTL auto-cleanup keeps hot data resident and cold data cheap, directly controlling cloud costs.

— Hybrid search with dense and sparse vectors plus RRF re-ranking catches both semantic meaning and exact keyword matches in a single query.

— Dynamic Schema is fast for prototyping but should be replaced with a strict Schema before production to avoid unpredictable query behavior.

— Consistency level 'Session' gives read-your-writes guarantees for the same client, which is usually sufficient and cheaper than Strong consistency.

Conclusions

Index selection is not a one-time decision but a function of scale: the right index at 50k vectors is wrong at 5 million, and the migration path is rarely planned upfront.

The gap between 'it works in a notebook' and 'it runs in production' for vector DBs is almost entirely operational — load, release, flush, and consistency tuning are invisible until they break.

Cost optimization in vector databases is fundamentally a memory management problem, not a compute problem, which flips the usual cloud-cost intuition.

Hybrid search with RRF re-ranking is underused relative to its value; combining dense and sparse vectors often fixes retrieval failures that prompt engineering cannot.

Concepts & terms

ANN (Approximate Nearest Neighbor)

A class of algorithms that trade a small amount of accuracy for large speed gains when searching high-dimensional vector spaces, as exact nearest-neighbor search becomes prohibitively slow beyond small datasets.

RRF (Reciprocal Rank Fusion)

A re-ranking algorithm that merges result lists from multiple search methods by scoring each item based on its reciprocal rank across the lists, producing a combined ranking without requiring score normalization.

IVF_FLAT

An index type that first clusters vectors using K-Means, then performs brute-force search only within the nearest clusters, reducing the search space dramatically while keeping accuracy above 95%.

HNSW (Hierarchical Navigable Small World)

A graph-based index that builds a multi-layer structure where top layers skip across the space and bottom layers perform fine-grained search, achieving very high recall and QPS at the cost of higher memory usage.

Product Quantization (PQ)

A compression technique that splits high-dimensional vectors into sub-vectors and quantizes each independently, achieving extreme memory reduction (up to 16×) at the cost of lower recall.

Source: juejin.cn ↗ Google Translate ↗ Backup ↗