Frontend · Backend · Interview

RAG Embedding, Hand-Rolled: Vectors, Cosine Similarity, and FastRAG's Dual-Protocol Implementation

By 我是Allen · Oct 26, 2026

Read original on juejin.cn ↗ Google Translate ↗ Alt translation

RAG retrieval quality hinges on embedding choices: mixing vector models invalidates distances, empty chunks can kill a batch, and a dimension mismatch shows up as an ES mapping error unless caught early. FastRAG's implementation shows how to validate and retry every step in the ingestion path.

Summary

The embedding step in a hand-built RAG pipeline maps each text chunk to a dense vector, and semantic proximity becomes spatial distance measured by cosine similarity. Word embedding history runs from word2vec and GloVe to contextual models like BERT, then to sentence embeddings and the bi-encoder architecture, which lets documents be pre-encoded offline while queries are encoded at search time.

FastRAG's implementation shows where the pitfalls live. The domain interface splits EmbedDocs and EmbedQuery because many models require a query prefix like 'query: ' for retrieval. The code adapts to two cloud protocols with a single protocol switch, batches non-empty chunks while preserving array positions, and leaves empty texts as nil because a zero vector makes ES reject the batch. Dimension mismatches are caught right after parsing the response, before they surface as obscure mapping errors. A single retry with fixed 100 ms delay protects the whole import from transient failures.

Model selection goes beyond domain fit, retrieval accuracy, language support, chunk length and model size; price and cloud-provider fit matter in production. Switching models means re-embedding the entire corpus, because vectors from different models live in different spaces even when dimensions match.

Takeaways

— Different embedding models produce vectors in different spaces; even with matching dimensions, query and document must use the same model.

— Cosine similarity compares the angle between vectors, so it is independent of text length.

— The IEmbedding interface separates EmbedDocs and EmbedQuery because models like bge-m3, E5, and GTE require a query prefix.

— Empty text chunks are filtered out and left as nil in the result, since a zero vector is rejected by ES's cosine similarity and would cause a batch write to return 400.

— A dimension mismatch between config and actual vector is validated immediately after parsing the response, avoiding a confusing ES mapping error later.

— FastRAG retries each embedding batch once with a fixed 100 ms delay; context cancellation aborts the retry.

— A medium-sized document involves dozens of batches, so a single bad batch can fail the whole import; retry is per-batch, not checkpointed.

— Each chunk computes two vectors: title and body; title vectors are used for reranking (low noise), body vectors for recall (full information).

— The content vector path includes the heading chain prepended by the chunker, so section context is embedded.

Conclusions

Splitting title and content into two vectors per chunk reflects a standard RAG trade-off: recall wants information-dense content vectors, reranking wants low-noise title vectors.

The aggressive validation at parse time (dimension, presence, count) implies embedding APIs can return inconsistent or malformed responses; trusting them blindly breaks downstream ingestion.

The bi-encoder architecture is the reason million-scale retrieval is feasible: document vectors are precomputed once, and only the query needs encoding at search time.

Separating query and document encoding is a detail often skipped; models like bge-m3, E5, and GTE need a query prefix, and mixing the two paths measurably hurts score.

Concepts & terms

Cosine similarity

A metric that measures the angle between two vectors; direction matters, magnitude is ignored, so it works for text of different lengths.

Bi-encoder

A model architecture that encodes a query and a document independently into two vectors, allowing document vectors to be precomputed and stored for fast retrieval.

Sentence embedding

A dense vector that represents an entire piece of text (a sentence or paragraph) as a single point in high-dimensional space, as opposed to token-level vectors.

Source: juejin.cn ↗ Google Translate ↗ Backup ↗