RAG Application Functional Boundaries

Alignment of Boundaries

What follows describes the capabilities of a standalone retrieval service (Retrieval Service), and does not include agent loops, recall evaluation, or multi-turn retrieval decisions (those live in the main agent). This RAG service is only responsible for: document ingestion pipeline + retrieval API + operations/permissions/evaluation; it returns raw chunks + metadata to the outside, and does not generate answers, does not decide how to search or how many times to search.

It is split into five major blocks: 【Document Ingestion & Parsing Pipeline】, 【Chunking & Index Management】, 【Core Online Retrieval Capabilities】, 【Enterprise Governance Capabilities】, 【Observability & Evaluation Operations】, plus advanced multimodal/knowledge-graph extensions.

1. Document Ingestion & Parsing Pipeline

This is the offline side, turning raw files into searchable content.

1. Multi-source connectors

   • Files: PDF, Word, Excel, PPT, Markdown, txt, epub, scanned documents
   • Data sources: object storage (MinIO/COS), local directories, web crawlers, API sync, databases, Git, OA/SharePoint/Jira, etc.
   • Incremental sync: scheduled polling, Webhook-triggered changes, message queue listening; only newly added/modified documents are processed, avoiding full re-scans.

2. Document parsing engine

   • Regular parsing: extract text, preserve heading hierarchy, table extraction, table of contents, page numbers, section position metadata.
   • OCR: recognize text in scanned documents and images; layout parsing for complex scenarios (distinguish body, table headers, captions, headers and footers).
   • Cleaning: remove watermarks, duplicate headers/footers, repair garbled text, filter blank paragraphs, normalize formatting.
   • Multimodal parsing: figure caption extraction, chart OCR, image-to-description (VL parsing).

3. Document version management

   • Multiple versions of the same document archived, with version switching and freezing of old-version indexes; document deletion/soft deletion and index cleanup.

2. Chunking, Embedding, Index Management

1. Multiple chunking strategies (configurable per knowledge base/document type)

   • Basic: fixed-length chunking, recursive chunking.
   • Advanced: heading-aware chunking, semantic chunking, table-independent chunking, section-boundary chunking, with overlapping windows.
   • Chunk-level metadata binding: document ID, file name, page number, section, uploader, department, time, tags, permission tags.

2. Embedding management

   • Embedding model routing; multiple embeddings can coexist (different libraries use different models).
   • Vector normalization, vector compression (PQ).
   • Batch re-embedding pipeline when switching models.

3. Indexing and storage

   • Vector database (Milvus/Qdrant/pgvector) + full-text database (ES/Elasticsearch) dual indexes.
   • Multi-tenant/Namespace isolation, knowledge base isolation, per-library switches.
   • Index lifecycle: hot index, cold archive, TTL, automatic cleanup of stale chunks.

3. Core Online Retrieval API Capabilities (the core APIs called by the main agent)

This is the tool interface your main agent calls; it only returns chunks, it does not generate answers.

1. Hybrid search

   • Dense vector retrieval (semantic) + BM25 sparse keyword retrieval + RRF fusion scoring (industry standard).
   • Pure vector, pure keyword, and hybrid modes are switchable.

2. Retrieval filtering

   • Metadata filtering: time range, document type, department, device number, regulation version, tags.
   • Document-level ACL permission filtering: the API takes user identity and automatically filters out chunks invisible to that user (prevents privilege escalation).

3. Rerank

   • First coarse recall topN (e.g., 20–50) → reranker model scores → return topK.
   • Supports toggling, custom rerankers, timeout degradation (if reranking fails, directly return the coarse recall results).

4. Result post-processing

   • Deduplication, near-duplicate merging, filtering low-score fragments.
   • Unified return structure: chunk text, source document ID, page number, section, score, original file link, metadata.

5. Cache, rate limiting, circuit breaking, degradation

   • Query cache (Redis; same query + same filter conditions reuse recall results, with expiration).
   • API rate limiting, concurrency control, timeout settings; automatic degradation strategy when the vector database/ES is abnormal.

6. Multi-knowledge-base routing

   • One retrieval API routes automatically to different knowledge bases based on route parameters (procedures library, defects library, historical solutions library).

✅ Distinction: Query Rewrite / Sub-question live in the main agent, not in this retrieval service.

4. Enterprise Governance & Security Capabilities (must add in production)

1. Permission system

   • Knowledge-base-level permissions, document-level permissions, user/role/department ACL.
   • API authentication: API Key, JWT, and SSO integration with the main business.

2. Content security

   • Review of ingested content, sensitive-word filtering on retrieval results, desensitization (phone numbers, ID numbers, confidential fields).

3. Knowledge base management console (Web)

   • Upload documents, view document list, preview, manually trigger re-parsing/re-chunking/re-indexing.
   • Tag management, knowledge base creation, switches, statistics (document count, chunk count, storage usage).
   • Manually test retrieval: enter a query, see recalled chunks, scores, reranking effect (for debugging).

5. Observability, Evaluation, Iteration Capabilities

1. End-to-end tracing

   • Every retrieval is instrumented: query, filter conditions, recall count, scores before/after reranking, latency, returned chunk IDs, user ID, knowledge base ID.
   • Integrates with LangFuse/LangSmith, connected with the upper agent's trace; one trace shows: agent calls retrieve → full internal retrieval flow.

2. Metric monitoring (Prometheus + Grafana)

   • P95/P99 latency, QPS, error rate, cache hit rate, recall count distribution, reranking latency.
   • Ingestion pipeline metrics: parse failure rate, vectorization failures, sync delay.

3. RAG evaluation module (offline evaluation)

   • Integrates Ragas and DeepEval: Context Recall, Context Precision, Faithfulness.
   • Test set management, batch evaluation, version comparison (compare effects before/after changing chunking, Embedding, reranker).
   • Human annotation entry: mark whether recall is relevant, feed back into the test set.

4. Feedback loop

   • Mark retrieval results as "relevant/irrelevant", used for later optimization of chunking, filtering, reranking.

6. Advanced Extension Capabilities (on demand, not default)

1. Multimodal retrieval: mixed image and text, image retrieval, table retrieval, image-text aligned retrieval after VL parsing.
2. GraphRAG knowledge graph retrieval: entity and relation extraction, subgraph retrieval, returned in parallel with vector retrieval (suitable for related queries about equipment, procedures, defects).
3. Summary index / small-summary assisted retrieval: build a separate index for chunk summaries (note: the original chunk is still returned to the agent; you cannot use only the summary).
4. Citation positioning: precise positioning of original PDF text (page number + coordinates; the front end can highlight the original text).

7. Key Boundaries, Avoid Coupling

✅ In the retrieval service: parsing, chunking, indexing, hybrid search, filtering, reranking, permission filtering, returning original fragments.
❌ Not in the retrieval service (put in the main agent): query rewriting, sub-question decomposition, deciding whether to retrieve, evaluating whether recall is sufficient, multi-turn loop retrieval, answer generation, hallucination checking.