Milvus Hybrid Search: BM25 Keyword Recall, Dense Vectors, and Cross-Encoder Reranking in One Schema
Python Practical Retrieval Optimization: BM25, Vectors, and Reranking
Official site: Milvus
Hybrid Retrieval and Cross-Encoder Fine-Ranking
Keyword retrieval catches proper nouns, semantic retrieval finds similar meanings; hybrid retrieval first expands recall, then Cross-Encoder fine-ranking sorts. Below, from deployment to invocation, we connect the knowledge ingestion and retrieval flow.
Flow: Start service → Create Collection → Write text → Hybrid recall → Fine-ranking
Core distinction:
TEXTEMBEDDINGcalls an embedding model to generate dense vectors;BM25generates sparse vectors in Milvus without calling an external model;RERANKreorders based on the question and candidate text.
🚀 1. Starting Milvus on a BaoTa Server
First install Docker and Docker Compose, and place the Milvus deployment configuration on the server. The screenshot red box marks the deployment directory:
Confirm Docker is available in the BaoTa terminal or SSH, then enter the deployment directory and start:
docker --version
docker compose version
cd /root/deploy
docker compose up -d
Check container status:
docker compose ps
Milvus, etcd, MinIO should be Up or healthy, Attu should be Up:
When first opening the Attu admin page http://server-address:3000, create an admin account first:
Password should be 8-128 characters and include at least three of the four types: uppercase, lowercase, digits, symbols. Save the account; after the first admin is created, public registration is closed.
If status is abnormal, view the logs:
docker compose logs --tail=100
Ports: 3000 for Attu, 19530 for application connections. Only open these to trusted IPs.
📦 2. Installing PyMilvus
PyMilvus is the Python client, installed in the application environment and used there; it does not start the Milvus service:
python3 -m pip install pymilvus
When using uv:
uv add pymilvus
Verify the installation:
python3 -c "from pymilvus import MilvusClient; print('PyMilvus 已安装')"
The client version must be compatible with the server. When connection fails, check the address, port, and authentication.
🧱 3. Preparing Connection Info
A Collection is a data collection, a Schema defines fields and types; an Entity is a record.
The Attu overview shows the version, deployment mode, and connection address:
Address field: milvus:19530 only applies within the same Docker network. Other machines use the reachable server IP or domain, and must ensure 19530 is open; the uri in code uses this address.
🔐 4. Configuring Model Credentials
In the BaoTa file manager, enter /root/deploy/milvus/ and merge the credentials into the Milvus server configuration. The embedding model and Cross-Encoder share this alias; BM25 does not need an external model. Do not commit or expose the real API key:
credential:
dashscope_apikey:
apikey: YOUR_DASHSCOPE_API_KEY
function:
textEmbedding:
providers:
dashscope:
credential: dashscope_apikey
rerank:
model:
providers:
ali:
credential: dashscope_apikey
Keep the existing configuration, save, then restart the service:
cd /root/deploy
docker compose restart
🧩 5. Wrapping the Vector Database and Calls
Below, in execution order, collection creation, writing, ranker creation, and hybrid retrieval are broken into small functions. First replace the connection address and confirm the vector dimension matches the embedding model output:
from uuid import uuid4
from pymilvus import (
AnnSearchRequest,
DataType,
Function,
FunctionType,
MilvusClient,
)
# The address must be reachable from the machine running Python.
client = MilvusClient(uri="http://your-server-address:19530")
collection_name = "knowledge_base"
vector_dim = 1024
def create_collection():
# Load the collection if it exists, avoiding duplicate creation.
if client.has_collection(collection_name):
client.load_collection(collection_name)
return
schema = client.create_schema(auto_id=False, enable_dynamic_field=False)
# Primary key, semantic vector, BM25 vector, plus the content and title returned on search.
schema.add_field("id", DataType.INT64, is_primary=True) # unique primary key per record
schema.add_field("dense", DataType.FLOAT_VECTOR, dim=vector_dim) # semantic vector
schema.add_field("sparse", DataType.SPARSE_FLOAT_VECTOR) # BM25 keyword vector
schema.add_field("h2", DataType.VARCHAR, max_length=128) # title, for display/filtering
# VARCHAR length counts bytes; Chinese content enables Chinese tokenization.
schema.add_field(
"content",
DataType.VARCHAR,
max_length=65535,
enable_analyzer=True,
analyzer_params={"type": "chinese"},
)
# Milvus generates the BM25 sparse vector when content is written.
schema.add_function(Function(
name="text_bm25_emb",
input_field_names=["content"],
output_field_names=["sparse"],
function_type=FunctionType.BM25,
))
# TEXTEMBEDDING calls DashScope through Milvus to convert content into a dense vector.
schema.add_function(Function(
name="text_embedding",
input_field_names=["content"],
output_field_names=["dense"],
function_type=FunctionType.TEXTEMBEDDING,
params={
"provider": "dashscope",
"model_name": "text-embedding-v3",
"dim": str(vector_dim),
},
))
# Dense vectors use COSINE; BM25 sparse vectors use an inverted index.
indexes = client.prepare_index_params()
indexes.add_index(field_name="dense", index_type="AUTOINDEX", metric_type="COSINE")
indexes.add_index(
field_name="sparse",
index_type="SPARSE_INVERTED_INDEX",
metric_type="BM25",
)
client.create_collection(
collection_name=collection_name,
schema=schema,
index_params=indexes,
)
client.load_collection(collection_name)
def insert_documents(documents: list[dict]):
# Each input has at least content; h2 is optional.
if not documents:
return []
# Create the collection before calling; each record uses a unique INT64 primary key.
create_collection()
# Milvus INT64 primary keys are signed integers, so take only the low 63 bits of the UUID.
ids = [uuid4().int & ((1 << 63) - 1) for _ in documents]
# Only original text and scalar fields are written; both vectors are auto-generated by the schema functions.
entities = [
{
"id": doc_id,
"h2": document.get("h2", ""),
"content": document["content"],
}
for doc_id, document in zip(ids, documents)
]
client.insert(collection_name=collection_name, data=entities)
return ids
def create_cross_encoder_ranker(query: str):
# A Cross-Encoder reads the question and candidate content together, recomputing relevance.
return Function(
name="cross_encoder_ranker",
input_field_names=["content"],
function_type=FunctionType.RERANK,
params={
"reranker": "model",
# provider and credential alias must match the server configuration.
"provider": "ali",
"model_name": "qwen3.7-text-rerank",
"queries": [query],
# At most 5 candidates per batch given to the model.
"max_client_batch_size": 5,
"credential": "dashscope_apikey",
},
)
def hybrid_search(query: str, fetch_k: int = 8, top_k: int = 3):
# Do not search on an empty query or before the collection exists.
query = query.strip()
if not query or not client.has_collection(collection_name):
return []
client.load_collection(collection_name)
# Each route recalls fetch_k candidates; the dense query calls the embedding model, BM25 is handled by Milvus.
dense_request = AnnSearchRequest(
data=[query], anns_field="dense", param={}, limit=fetch_k
)
sparse_request = AnnSearchRequest(
data=[query], anns_field="sparse", param={}, limit=fetch_k
)
# The ranker handles fusion and fine-ranking; top_k is the final return count.
results = client.hybrid_search(
collection_name=collection_name,
reqs=[dense_request, sparse_request],
ranker=create_cross_encoder_ranker(query),
limit=top_k,
output_fields=["content", "h2"],
)
# data has only one query, so return the first set of hits.
return results[0] if results else []
# Prepare the collection and store original text on write; Milvus auto-generates dense and sparse vectors.
insert_documents([
{"h2": "连接排查", "content": "蓝牙设备反复断连时,可以先检查电量,再删除配对记录并重新连接。"},
{"h2": "充电问题", "content": "耳机无法充电时,先清理充电触点,并确认充电盒电量充足。"},
])
hits = hybrid_search("无线耳机总是断开连接,先检查哪些地方?")
for hit in hits:
# entity is the matched field; distance is the relevance score Milvus returns.
entity = hit["entity"]
print(f"{hit['distance']:.4f} | {entity['h2']} | {entity['content']}")
Code order: create_collection() builds the collection → insert_documents() writes the content → hybrid_search() retrieves. Milvus's TEXTEMBEDDING generates dense vectors at ingestion and query time; BM25 does not call an external model. Each route recalls fetch_k items, then RERANK fine-ranks and returns top_k items.
⭐ 6. Troubleshooting Focus
When results are poor, first confirm the relevant content can enter the candidate set; if candidates are correct but ranked low, check the credentials, rerank model, and candidate count. Increasing fetch_k expands the rerank scope but also adds latency; top_k controls the final return count.
🌱 Conclusion
At this point, Milvus can receive text, auto-generate vectors, and combine keyword and semantic recall at retrieval time, then sort with Cross-Encoder. When integrating your own knowledge, prepare the same content, h2 fields, and confirm the vector dimension, model credentials, and server address match.
Top 3 of 7 from juejin.cn, machine-translated. The original thread is authoritative.
Looking for an independent developer to build a custom TikTok women's fashion AI material pipeline: needs 3D fingerprint deduplication (image + text + shot timing), ComfyUI batch image generation, data feedback and scoring loop, deliverable source code — not a Codex node package. Anyone who has done similar projects, DM me with case studies.
[striking emoji]
?
Learned something, thanks for sharing!
[fighting][fighting][fighting]
Also waiting for cost-effective generation APIs~ If there's a later need for batch video material production, the MiniMax H3 API is worth referencing: cloud-based calls, built-in stereo sound, reference billing around 0.3 yuan/minute, suitable for running at scale. Features and pricing subject to official announcements.