Aligning the RAG Boundary: What Stays in the Retrieval Service and What Moves to the Agent

RAG systems fail in production when retrieval services quietly absorb agent responsibilities such as query rewriting and answer generation; this scope definition gives teams a concrete checklist for where to put each function. That separation keeps retrieval fast, auditable, and permission-safe while the agent handles the reasoning loop.

A RAG application works cleanly only when the retrieval service and the agent have hard boundaries. A standalone retrieval service handles the offline document pipeline, chunking and embedding, hybrid retrieval, reranking, permissions, and evaluation. It returns raw chunks with metadata, never generated answers. The agent above it owns query rewriting, sub-question decomposition, deciding whether to search, multi-turn retrieval, and hallucination checks.

The service separates into five required blocks: ingestion connectors and parsing, index management, online retrieval APIs, enterprise governance, and observability. Each has specific production defaults: dual vector and full-text indexes, hybrid BM25 plus dense retrieval with RRF fusion, ACL filtering at the document level, Redis query caching, and reranking with timeout degradation. The enterprise layer adds content moderation, desensitization, and a web console for manual debugging.

Advanced capabilities such as GraphRAG and multimodal retrieval are optional, not baseline. The operational rule is to avoid coupling: keep parsing, chunking, retrieval, and permission filtering inside the retrieval service, and push query rewriting, answer generation, and loop decisions into the main agent.