Milvus from Prototype to Production: A Complete Walkthrough for RAG Memory
RAG Performance Always Stuck on Retrieval? Start by Understanding the Milvus Vector Database
While building the memory module for an Agent, I walked through the complete Milvus integration process. This article uses an AI diary project as an example to sort out the key technical points.
1. What Problem Does a Vector Database Solve?
Traditional databases rely on exact keyword matching. Searching for "outdoor activities" won't find semantically related content like "mountain climbing" or "walking in the park." A vector database converts text into high-dimensional vectors using an Embedding model and measures semantic similarity by vector distance, enabling "search by meaning."
In a RAG architecture, the vector database handles the knowledge retrieval step and serves as the foundation for an AI Agent's long-term memory.
Core Concept Mapping
| Concept | Analogy | Description |
|---|---|---|
| Collection | Table | A data table |
| Entity | Row | A single record |
| Field | Column | A field, either scalar or vector type |
| Partition | Partitioned Table | Logical isolation, used for multi-tenancy or time-based archiving |
| Index | Index | Accelerates ANN (Approximate Nearest Neighbor) search |
2. Environment Setup
pnpm add @zilliz/milvus2-sdk-node @langchain/openai dotenv
import { MilvusClient } from '@zilliz/milvus2-sdk-node';
import { OpenAIEmbeddings } from '@langchain/openai';
// Using Alibaba Tongyi text-embedding-v2 as an example, dimension 1536
const embeddings = new OpenAIEmbeddings({
apiKey: process.env.OPENAI_API_KEY,
model: 'text-embedding-v2',
configuration: { baseURL: process.env.OPENAI_BASE_URL },
dimensions: 1536,
});
const client = new MilvusClient({
address: process.env.MILVUS_ADDRESS,
token: process.env.MILVUS_TOKEN,
maxRetries: 3,
timeout: 30000,
});
async function getEmbeddings(text) {
return await embeddings.embedQuery(text);
}
Zilliz Cloud is maintenance-free and ready to use out of the box. For self-hosting, start with a single Docker Compose command; the default address is localhost:19530 with no token required.
3. Schema Design and Collection Management
3.1 Field Types
| Type | Use Case |
|---|---|
VarChar |
id, content, date, mood |
Int64 / Float / Bool |
Numeric and boolean scalars |
JSON |
Flexible metadata |
Array |
Tags |
FloatVector |
Content Embedding |
BinaryVector / SparseFloatVector |
Image fingerprints / keyword sparse vectors |
3.2 Creating an AI Diary Collection
await client.createCollection({
collection_name: 'ai_diary',
fields: [
{ name: 'id', data_type: DataType.VarChar, max_length: 50, is_primary_key: true },
{ name: 'vector', data_type: DataType.FloatVector, dim: 1536 },
{ name: 'content', data_type: DataType.VarChar, max_length: 5000 },
{ name: 'date', data_type: DataType.VarChar, max_length: 50 },
{ name: 'mood', data_type: DataType.VarChar, max_length: 50 },
{ name: 'tags', data_type: DataType.Array, element_type: DataType.VarChar, max_capacity: 10, max_length: 50 },
],
});
Milvus also supports Dynamic Schema (enableDynamicField: true), where undeclared fields are automatically placed into a JSON column. This is suitable for rapid prototyping iterations, but it's recommended to switch to a strict Schema for production.
3.3 Lifecycle Operations
await client.hasCollection({ collection_name: 'ai_diary' });
await client.describeCollection({ collection_name: 'ai_diary' });
await client.showCollections();
await client.alterCollection({ collection_name: 'ai_diary', properties: { 'collection.ttl.seconds': '604800' } });
await client.dropCollection({ collection_name: 'ai_diary' });
3.4 Partition
await client.createPartition({ collection_name: 'ai_diary', partition_name: '2026_01' });
await client.insert({ collection_name: 'ai_diary', partition_name: '2026_01', data: [...] });
await client.search({ collection_name: 'ai_diary', partition_names: ['2026_01'], vector, limit: 10 });
Limiting searches to specific partitions can drastically reduce the candidate set, significantly improving QPS.
4. Vector Indexes — The Core of Performance
4.1 Index Selection Comparison
| Index | Principle | Accuracy | Memory | Suitable Scale |
|---|---|---|---|---|
| FLAT | Brute-force search | 100% | Low | <100k |
| IVF_FLAT | K-Means + brute-force on candidate set | >95% | Medium | Millions |
| IVF_SQ8 | IVF + 8-bit quantization | >92% | Low (4× compression) | Millions, memory-constrained |
| IVF_PQ | IVF + Product Quantization | >85% | Very Low (16×) | Tens of millions |
| HNSW | Hierarchical graph search | >98% | High | Millions, high QPS |
| DISKANN | SSD-based | >95% | Disk | Billions |
4.2 Index Creation
// Default choice: IVF_FLAT + COSINE
await client.createIndex({
collection_name: 'ai_diary',
field_name: 'vector',
index_type: 'IVF_FLAT',
metric_type: 'COSINE',
params: { nlist: 1536 },
});
// HNSW — High recall, high QPS, trades memory for performance
await client.createIndex({
collection_name: 'ai_diary', field_name: 'vector',
index_type: 'HNSW', metric_type: 'COSINE',
params: { M: 16, efConstruction: 200 },
});
// IVF_SQ8 — Compresses vectors to 1/4 size
await client.createIndex({
collection_name: 'ai_diary', field_name: 'vector',
index_type: 'IVF_SQ8', metric_type: 'COSINE',
params: { nlist: 1024 },
});
Decision Logic: Use FLAT for <100k vectors; default to IVF_FLAT for millions; use IVF_SQ8 or IVF_PQ when memory is constrained; use HNSW for high QPS; use DISKANN for billions.
4.3 Similarity Metrics
| Method | Formula | Applicable |
|---|---|---|
| COSINE | Cosine Similarity | Text semantics (most common) |
| L2 | Euclidean Distance | Unnormalized vectors |
| IP | Inner Product | Normalized vectors (fastest computation) |
| HAMMING / JACCARD | Hamming/Jaccard | BinaryVector |
Note: If vectors are already L2-normalized, IP and COSINE are mathematically equivalent, but IP has lower computational overhead. The text-embedding-v2 model used in this project does not normalize, so COSINE is used.
5. Data Operations
Insert
const diaryData = [
{ id: 'diary_001', content: 'The weather was great today, went for a walk in the park...', date: '2026-01-10', mood: 'happy', tags: ['life', 'walk'] },
{ id: 'diary_002', content: 'Very busy at work today, completed an important project milestone...', date: '2026-01-11', mood: 'excited', tags: ['work', 'achievement'] },
{ id: 'diary_003', content: 'Went hiking with friends over the weekend, the weather was great...', date: '2026-01-12', mood: 'relaxed', tags: ['outdoor', 'friends'] },
{ id: 'diary_004', content: 'Learned about the Milvus vector database today...', date: '2026-01-12', mood: 'curious', tags: ['study', 'tech'] },
{ id: 'diary_005', content: 'Cooked a big dinner tonight...', date: '2026-01-13', mood: 'proud', tags: ['food', 'family'] },
];
const data = await Promise.all(diaryData.map(async d => ({ ...d, vector: await getEmbeddings(d.content) })));
const { insert_cnt } = await client.insert({ collection_name: 'ai_diary', data });
Upsert / Query / Delete
// Upsert: Updates if exists, inserts if not
await client.upsert({ collection_name: 'ai_diary', data: [{ id: 'diary_001', vector: newVec, content: '...' }] });
// Query: Exact match on scalar conditions, does not use vectors
await client.query({ collection_name: 'ai_diary', expr: 'mood == "happy"', output_fields: ['id', 'content'], limit: 100 });
// Delete
await client.delete({ collection_name: 'ai_diary', expr: 'id in ["diary_003"]' });
await client.delete({ collection_name: 'ai_diary', expr: 'date < "2026-01-01"' });
Query vs Search: Query performs exact filtering on scalars and is fast; Search performs vector ANN search and can match semantically similar results.
6. Vector Search
Basic Semantic Search
await client.loadCollection({ collection_name: 'ai_diary' });
const query = 'I want to see diary entries about outdoor activities';
const queryVector = await getEmbeddings(query);
const { results } = await client.search({
collection_name: 'ai_diary',
vector: queryVector,
limit: 3,
metric_type: 'COSINE',
output_fields: ['id', 'content', 'date', 'mood', 'tags'],
});
Searching for "outdoor activities" will prioritize returning diary_003 (hiking) and diary_001 (walking in the park), even though the original text does not contain the exact phrase "outdoor activities."
Scalar-Filtered Search
// Vector + condition combination, standard usage in production
await client.search({
collection_name: 'ai_diary',
vector: queryVector,
limit: 10,
filter: 'array_contains(tags, "outdoor") and date >= "2026-01-10"',
output_fields: ['id', 'content', 'tags'],
});
Supported expressions: ==, !=, >, >=, <, <= / and, or, not / in [...] / like "prefix%" / array_contains() / metadata["key"].
Range Search and Pagination
// Search by similarity threshold
await client.search({ ..., params: { radius: 0.8, range_filter: 0.5 } });
// Pagination
await client.search({ ..., limit: 10, offset: 0 });
await client.search({ ..., limit: 10, offset: 10 });
Multi-Vector Hybrid Search (2.4+)
Use dense vectors (semantics) and sparse vectors (keywords) simultaneously for multi-recall, with RRF re-ranking:
await client.hybridSearch({
collection_name: 'ai_diary',
searches: [
{ vector: denseVec, anns_field: 'dense_vector', metric_type: 'COSINE', limit: 100 },
{ vector: sparseVec, anns_field: 'sparse_vector', metric_type: 'IP', limit: 100 },
],
rerank: { strategy: 'rrf', params: { k: 60 } },
limit: 10,
output_fields: ['id', 'content'],
});
7. Memory Management and Consistency
A Collection must be loaded into QueryNode memory before it can be searched, and released when done:
await client.loadCollection({ collection_name: 'ai_diary', replica_number: 2 });
await client.releaseCollection({ collection_name: 'ai_diary' });
Four consistency levels:
| Level | Semantics | Scenario |
|---|---|---|
| Strong | Linearizable, guarantees reading the latest | Finance, inventory |
| Session | Same-client read-your-writes | Query immediately after write |
| Bounded | Tolerates delay within a specified window | General business |
| Eventually | Eventual consistency, may be stale | Recommendations, analytics |
await client.insert({ ..., consistency_level: 'Session' });
8. Production Considerations
Error Retries: The SDK has built-in maxRetries. It's recommended to add an additional layer of backoff retry for rate_limit at the business logic level.
Batch Writes: Submit in batches of 1000 records, and explicitly call flush at the end to ensure data is persisted to disk.
Model Selection:
| Model | Dimensions | Characteristics |
|---|---|---|
| text-embedding-3-small (OpenAI) | 1536 | General-purpose, balanced |
| text-embedding-v2 (Alibaba) | 1536 | Strong Chinese semantics |
| BGE-M3 (BAAI) | 1024 | Open-source, dense + sparse |
| M3E-large | 1024 | Open-source, lightweight Chinese |
Once chosen, do not change models lightly — switching models means re-embedding and re-inserting all vectors.
Cost Control: A dimension of 1536 is sufficient. Use IVF_SQ8 to compress memory by 75% for non-core scenarios. Partition by time and use TTL for automatic cleanup. Release cold data and keep hot data resident in memory.
9. Summary
Using the AI diary project as a thread, this article covered the core chain from Milvus integration to production:
- Schema: Field types, strong/dynamic Schema, Partition
- Indexes: The progressive path from FLAT → IVF_FLAT → HNSW → IVF_SQ8 → IVFPQ → DISKANN
- Operations: Insert / Upsert / Query / Delete
- Search: Semantic search, scalar filtering, range search, pagination, multi-vector hybrid + RRF
- Reliability: Consistency levels and memory management
- Engineering: Retries, batch writes, model selection, cost optimization
A vector database is the memory foundation of RAG. Understanding Milvus means understanding the underlying logic of the AI retrieval chain.