跪拜 Guibai
← Back to the summary

A FastAPI Monolith Runs an Enterprise Knowledge Base with RAG, KAG, and 9 MySQL Databases

Enterprise Intelligent Knowledge Base Python Backend in Practice: FastAPI + RAG + KAG Enterprise AI Project

1. Project Overview

The enterprise intelligent knowledge base Python backend is a complete enterprise-grade intelligent knowledge management platform backend service. Built on the FastAPI framework and using a modular APIRouter architecture, it integrates document management, distributed file storage, full-text search, artificial intelligence (RAG/KAG), knowledge graphs, and other cutting-edge technologies, providing enterprises with a full-chain knowledge management solution from document creation, storage, retrieval, sharing to intelligent Q&A.

1.1 Project Positioning

Dimension Description
Project type Enterprise-grade intelligent knowledge management platform
Architecture pattern Module-based Monolith
Backend technology FastAPI + Python 3.12+ + SQLAlchemy 2.0
Databases MySQL 8.0 × 9 + Redis + Elasticsearch + MongoDB + Neo4j
Message queue RabbitMQ + Celery
File storage S3-compatible (MinIO / RustFS)
AI capabilities LLM chat + RAG retrieval-augmented generation + KAG knowledge-augmented generation

More hands-on projects at the Java Commando network: susan.net.cn/project

1.2 System Architecture Diagram

image.png

2. Project Architecture

2.1 Modular Monolith Architecture

The project uses a module-based monolith architecture: a single process runs the application, internally split into independent apps/ modules by business domain, with route isolation achieved through FastAPI's APIRouter mechanism.

┌──────────────────────────────────────┐
│           FastAPI Application         │
│              (main process :8080)     │
│                                       │
│  ┌──────────┐  ┌──────────┐          │
│  │ Gateway  │  │   CORS   │  middleware│
│  │ Auth     │  │  Logger  │          │
│  └──────────┘  └──────────┘          │
│                                       │
│  ┌──────────────┐  ┌──────────────┐  │
│  │ apps/        │  │ apps/        │  │
│  │ user_auth    │  │ document     │  │
│  │ ┌──────────┐ │  │ ┌──────────┐ │  │
│  │ │ schemas  │ │  │ │ schemas  │ │  │
│  │ │ services │ │  │ │ services │ │  │
│  │ │ routers  │ │  │ │ routers  │ │  │
│  │ │ models   │ │  │ │ models   │ │  │
│  │ └──────────┘ │  │ └──────────┘ │  │
│  └──────────────┘  └──────────────┘  │
│                                       │
│  ┌──────────────┐  ┌──────────────┐  │
│  │ apps/file    │  │ apps/search  │  │
│  └──────────────┘  └──────────────┘  │
│                                       │
│  ┌──────────────┐  ┌──────────────┐  │
│  │ apps/ai      │  │ apps/graph   │  │
│  └──────────────┘  └──────────────┘  │
│                                       │
│  ┌──────────────┐  ┌──────────────┐  │
│  │ apps/        │  │ apps/        │  │
│  │ foundation   │  │ statistics   │  │
│  └──────────────┘  └──────────────┘  │
│                                       │
│  ┌────────────────────────────────┐  │
│  │          common/               │  │
│  │  database | redis | mongo      │  │
│  │  rabbitmq | security | ...     │  │
│  └────────────────────────────────┘  │
└──────────────────────────────────────┘

Architecture advantages:

2.2 Module List

Module Directory Core responsibility
gateway apps/gateway/ Unified entry, route rewriting, CORS, JWT auth whitelist, request logging
common common/ Shared infrastructure: database management, Redis, MongoDB, RabbitMQ, JWT, snowflake ID
user_auth apps/user_auth/ User authentication, role permissions (RBAC), team management, menu management
document apps/document/ Document CRUD, categories and tags, comment review, version management, favorites and sharing, PDF export
file apps/file/ File upload/download, S3 storage, file preview
search apps/search/ Full-text search, hybrid search (BM25 + vectors), search history
statistics apps/statistics/ System statistics, data overview, trend analysis
graph apps/graph/ Knowledge graph: entity relations, path analysis, community discovery
foundation apps/foundation/ System configuration, dictionary management, operation logs, WebSocket notifications
ai apps/ai/ AI chat, RAG Q&A, KAG knowledge enhancement, AI writing, document intelligence

2.3 Tech Stack

Stack Version Purpose
FastAPI >=0.115 Web framework
Uvicorn >=0.30 ASGI server
SQLAlchemy 2.0 >=2.0 ORM (async)
asyncmy >=0.2.9 MySQL async driver
Alembic >=1.13 Database migrations
Redis (hiredis) >=5.0 Cache
Elasticsearch (async) >=8.0 Full-text search
Motor + PyMongo >=3.5 MongoDB async client
Neo4j >=5.20 Knowledge graph
aio-pika >=9.4 RabbitMQ AMQP
Celery >=5.4 Async task queue
python-jose >=3.3 JWT handling
passlib (bcrypt) >=1.7 Password hashing
boto3 / aioboto3 >=1.34 S3 storage
python-docx >=1.0 Word document parsing
openpyxl >=3.1 Excel parsing
pymupdf >=1.24 PDF parsing + Markdown conversion
weasyprint >=60.0 PDF export
loguru >=0.7 Logging
pydantic v2 >=2.0 Data validation + config management
httpx >=0.27 HTTP client

3. Database Design

3.1 Database Distribution

Database Module Core tables
kb_user user_auth user, role, permission, team, menu
kb_document document document, category, tag, comment, review, version, favorite, share
kb_search search search_history, search_index_config
kb_file file file_info, file_category
kb_ai ai ai_conversation, ai_message, ai_feedback, ai_prompt_template
kb_statistics statistics dashboard_config, statistics_snapshot
kb_graph graph graph_node, graph_relation, graph_config
kb_foundation foundation sys_config, sys_dict, operation_log, notification
kb_common global shared common config and cross-database views

3.2 Extended Storage

Storage Purpose
MongoDB (knowledge_base) Document body content (document_content), editor auto-save history (auto_save_history)
Elasticsearch Document full-text index + vector embedding storage
Neo4j Knowledge graph entity nodes and relations
MinIO / S3 Uploaded files and images

3.3 Database Connection Management

common/database.py implements a DatabaseManager singleton that manages async connection pools for the 9 MySQL databases:

4. Core Feature Modules

4.1 Gateway Middleware (apps/gateway)

Gateway middleware handles request preprocessing and route distribution:

Feature Implementation
Path rewriting GatewayPathRewriter middleware, compatible with the /api/xxx route prefix; the front end need not be aware of backend sub-paths
CORS Global cross-origin configuration, supporting * or specified domains
JWT auth AuthMiddleware; whitelisted paths (17 public endpoints) skip validation
Request logging LoggingMiddleware, recording request method, path, status code, and duration

4.2 User Authentication and Permissions (apps/user_auth)

4.3 Document Management (apps/document)

Covers the full document lifecycle:

create → edit → submit for review → review → approve → publish → archive
  ↑                ↓
  └── rejected (back to editing) ──┘

Core features:

4.4 File Management (apps/file)

4.5 Intelligent Search (apps/search)

4.6 AI Services (apps/ai)

4.7 Knowledge Graph (apps/graph)

4.8 Foundation Services (apps/foundation)

4.9 Statistics and Analytics (apps/statistics)

5. Common Infrastructure (common/)

5.1 Component List

Module Function
database.py Async connection pool management for 9 MySQL databases, lazy initialization
redis.py Redis singleton connection pool (pool size 50), JWT cache, verification code storage
mongo.py MongoDB Motor async client, lazy initialization
rabbitmq.py RabbitMQ aio-pika async connection, Topic Exchange declaration
security.py JWT HS256 token create/verify/refresh, BCrypt password hashing
id_generator.py 64-bit snowflake ID, 41-bit timestamp + 12-bit sequence
base_model.py SQLAlchemy ORM base classes, timestamp mixin, soft-delete mixin, audit mixin
result.py Result[T] generic unified response format
exceptions.py Business exceptions + global exception handlers
deps.py FastAPI dependency injection (current user, permission checks, pagination params)
json_response.py Long-integer-safe JSON response (prevents front-end precision loss)
serializer.py Model serialization utilities
enums.py / constants.py Enum definitions + application constants

5.2 Key Design

Async-first: all external I/O (MySQL, Redis, MongoDB, ES, RabbitMQ, S3) uses async/await, so a single process handles high concurrency efficiently.

Snowflake ID: 64-bit distributed unique ID, start timestamp 2024-01-01, datacenter ID 1, at most 4096 IDs per millisecond, with clock-rollback tolerance.

Unified response format: all APIs return the Result[T] format:

{
  "code": 200,
  "message": "success",
  "data": { ... }
}

6. API Endpoint Overview

6.1 Endpoint Distribution

Module Route prefix Main endpoints
user_auth /api/auth, /api/users, /api/roles, /api/permissions, /api/teams, /api/menus Login, register, token refresh, user/role/permission/team/menu CRUD
document /api/documents, /api/categories, /api/tags, /api/comments, /api/reviews Document CRUD, categories, tags, comments, review, versions, favorites, sharing, upload parsing, PDF export
file /api/files File upload, download, preview, category management
search /api/search Keyword search, hybrid search, search history, index rebuild
statistics /api/statistics Dashboard, trends, hotspot data
graph /api/graph Nodes, relations, paths, communities, graph building
foundation /api/configs, /api/dicts, /api/logs, /api/notifications System config, dictionaries, operation logs, notification management
ai /api/ai Chat, RAG Q&A, KAG reasoning, AI writing, document intelligence
WebSocket /ws/notification Real-time notification push

6.2 API Documentation

After startup, interactive API documentation is available at:

7. Project Directory Structure

knowledge-base-python/
├── main.py                          # App entry (FastAPI instance + lifecycle management)
├── config.py                        # Global config (pydantic-settings, loaded from .env)
├── pyproject.toml                   # Project metadata + dependencies
├── uv.lock                          # UV dependency lock file
├── docker-compose.yml               # Infrastructure container orchestration
├── .env.example                     # Environment variable template
├── .gitignore
├── alembic.ini                      # Database migration config
│
├── common/                          # Shared infrastructure (13 modules)
│   ├── database.py                  # Connection pool management for 9 MySQL databases
│   ├── redis.py                     # Redis manager
│   ├── mongo.py                     # MongoDB manager
│   ├── rabbitmq.py                  # RabbitMQ manager
│   ├── security.py                  # JWT + BCrypt passwords
│   ├── id_generator.py              # Snowflake ID generator
│   ├── base_model.py                # ORM base classes
│   ├── result.py                    # Result[T] unified response
│   ├── exceptions.py                # Business exceptions + global exception handlers
│   ├── deps.py                      # Dependency injection
│   ├── json_response.py             # Long-integer-safe JSON
│   ├── serializer.py                # Serialization utilities
│   └── enums.py / constants.py      # Enums and constants
│
├── apps/                            # Business modules (microservice-style directories)
│   ├── gateway/                     # Gateway middleware
│   │   └── middleware.py
│   │
│   ├── user_auth/                   # User authentication (RBAC)
│   │   ├── schemas/                 # Pydantic request/response models
│   │   ├── services/                # Business logic layer
│   │   ├── routers/                 # API routes
│   │   └── models.py                # SQLAlchemy ORM models
│   │
│   ├── document/                    # Document management
│   │   ├── schemas/
│   │   ├── services/
│   │   │   ├── document_service.py      # Document CRUD + workflow
│   │   │   ├── file_parser_service.py   # PDF/DOCX/XLSX → Markdown
│   │   │   └── pdf_export_service.py    # Markdown → PDF
│   │   ├── routers/
│   │   └── models.py
│   │
│   ├── file/                        # File management
│   │   ├── schemas/
│   │   ├── services/
│   │   ├── routers/
│   │   ├── models.py
│   │   └── storage.py               # S3 storage abstraction
│   │
│   ├── search/                      # Search service
│   ├── graph/                       # Knowledge graph
│   ├── foundation/                  # Foundation services + WebSocket
│   ├── statistics/                  # Statistics and analytics
│   └── ai/                          # AI services
│
├── alembic/                         # Database migrations
├── scripts/                         # Utility scripts
├── docs/                            # Project documentation
└── sql/                             # Database initialization scripts

8. Technical Highlights

8.1 Fully Async I/O

All external calls (MySQL, Redis, MongoDB, ES, RabbitMQ, S3) use Python async/await async drivers, so a single process handles high-concurrency requests, with resource utilization far above a thread model.

8.2 Modular Design

Each apps/ module is self-contained with a three-layer architecture (schemas / services / routers), can be independently developed and tested, and can be smoothly split into independent microservices in the future.

8.3 Smart PDF/DOCX/XLSX File Parsing

file_parser_service.py implements a complete binary-file → Markdown conversion pipeline:

Format Parse engine Capabilities
PDF pymupdf (fitz) Text extraction, font styles (heading levels/bold/monospace), table detection (find_tables()), image extraction and upload to S3, code-block detection, paragraph boundary analysis
DOCX python-docx Paragraph style mapping (Heading 1-6 → #-######), inline formatting (bold/italic → **/*), tables → Markdown tables, embedded image extraction
XLSX openpyxl Worksheet → Markdown table, merged-cell handling

8.4 Markdown → PDF Export

pdf_export_service.py implements professional PDF export based on weasyprint:

8.5 WebSocket Real-Time Notifications

WebSocket endpoints compatible with the SockJS transport protocol support real-time push of review notifications, system messages, etc., with a 30-second heartbeat keeping connections alive.

8.6 Multi-AI-Model Adaptation

Multiple LLM providers (Tongyi Qianwen, DeepSeek) are accessed uniformly through an adapter pattern, with a 4-tier fallback strategy ensuring availability: primary model → retry → backup model → fallback response.

9. Quick Start

9.1 Environment Requirements

Component Version Notes
Python >= 3.12 3.13 recommended
UV latest Python package manager
Docker Desktop latest Run infrastructure containers

9.2 Startup Steps

# 1. Enter the project directory
cd knowledge-base-python

# 2. Start infrastructure (MySQL, Redis, ES, MongoDB, Neo4j, MinIO, RabbitMQ)
docker compose up -d

# 3. Create databases (9 MySQL databases)
docker exec -i kb-mysql mysql -uroot -proot123456 < sql/00_create_databases.sql

# 4. Configure environment variables
cp .env.example .env
# Edit the .env file, update database passwords, Redis/ES/Mongo addresses, etc.

# 5. Install Python dependencies
uv sync
uv pip install pymupdf

# 6. Run database migrations (create tables)
uv run alembic upgrade head

# 7. Start the backend service
uv run uvicorn main:app --host 0.0.0.0 --port 8080 --reload

9.3 Verification

Open http://127.0.0.1:8080/doc.html in the browser; seeing the Swagger API documentation means startup succeeded.

10. Comparison with the Java Backend

This project is fully API-compatible with the Java (Spring Cloud) backend of the same system, sharing the same databases and infrastructure. The two are positioned differently:

Dimension Java Backend Python Backend
Framework Spring Boot 3.2 + Spring Cloud 2023.0 FastAPI + Uvicorn
Architecture Microservices (8 independent processes + Nacos registry) Modular monolith (single process, APIRouter split)
Ports Separate ports per service (8080-8089) Unified port 8080
API endpoints ~197 One-to-one correspondence, fully compatible paths
ORM MyBatis Plus SQLAlchemy 2.0 Async
Databases MySQL × 9 + Mongo + ES + Neo4j Same schema, fully reused
Authentication JWT + Spring Security JWT + custom middleware
Password encryption BCrypt BCrypt (fully compatible, tokens interchangeable)
Message queue RabbitMQ (Spring AMQP) RabbitMQ (aio-pika)
Deployment Each service a separate JAR Single uvicorn process
Startup speed JVM cold start 30s+ Python cold start < 5s
Memory usage 512MB+ per service, 4GB+ total 200-500MB single process
PDF parsing Apache PDFBox + Tika pymupdf
DOCX parsing Apache POI python-docx
XLSX parsing Apache POI openpyxl
PDF export Apache PDFBox weasyprint
Video transcoding FFmpeg Planned
PPTX parsing Apache POI Planned

Selection advice:

More hands-on projects at the Java Commando network: susan.net.cn/project

11. System Screenshots

Below are actual screenshots from the running system, showing business capabilities fully implemented by the backend.

image.png

image.png

image.png

image.png

image.png

image.png

image.png

image.png