跪拜 Guibai
← Back to the summary

CodeWiki Uses Recursive Agents to Generate Docs for Million-Line Codebases

Daily Open-Source Project (Issue #170): CodeWiki — ACL 2026 Paper-Level Automatic Repository Documentation Generation, Recursive Multi-Agent Architecture

Introduction

"CodeWiki generates documentation for itself."

This is the 170th article in the "Daily Open-Source Project" series. Today's protagonist is CodeWiki — an automatic repository-level documentation generation framework open-sourced by the AI4Code team at FPT Software (Vietnam's largest IT company), and accepted at ACL 2026 (Annual Meeting of the Association for Computational Linguistics).

Repository documentation is a problem that hasn't been truly solved for decades. Function-level comments solve "what does this function do," but cross-file, cross-module architectural understanding — "why is this component here," "which layers does this data flow pass through" — has never had a systematic solution.

CodeWiki's approach: use a recursive multi-agent architecture to handle this scale problem. Tree-sitter parses ASTs to build a dependency graph, topological sorting finds the processing order, bottom-level modules are processed first and aggregated upward, and modules too complex to fit in a single pass automatically spawn child agents. The final output is complete Markdown documentation with Mermaid architecture diagrams.

What You Will Learn

Prerequisites

Project Background

Why Repository Documentation Is Hard

Function comment generation is a solved problem — GitHub Copilot and various AI-assisted tools can do it. The difficulty lies at higher levels:

Function-level (solved):
"This function receives a user_id, queries the database, and returns a user object"

Module-level (harder):
"This authentication module depends on user_service and cache_layer,
 verifies via JWT, and falls back to session verification on failure"

Repository-level (what CodeWiki aims to solve):
"What is the overall architecture of this codebase?
 How does data flow from the API layer to the storage layer?
 What are the dependency relationships between modules?"

The difficulty of repository-level understanding: dependency relationships are cross-file, architectural descriptions require a global view, but large codebases far exceed a single LLM's context window.

Author/Team Introduction

Project Data

Core Architecture: Three-Stage Pipeline

Stage 1: Repository Analysis (AST + Dependency Graph)

# CodeWiki uses Tree-sitter to parse all source files
# Extracts: functions, classes, cross-language dependency relationships
# Normalizes to depends_on relationships, builds a directed graph G=(V, E)

Codebase
  ↓
Tree-sitter AST parsing (supports 9 languages)
  ↓
Identify: function definitions, class definitions, module imports
  ↓
Normalize cross-file dependencies into a depends_on directed graph
  ↓
Topological sorting → find zero-indegree nodes (leaf modules with no dependencies)

The significance of the dependency graph: A depends_on B means understanding A requires understanding B first. Topological sorting gives the processing order — process dependencies first, then the modules that depend on them.

Stage 2: Recursive Multi-Agent Documentation Generation

This is CodeWiki's most core design.

Problem with ordinary LLMs processing codebases:

Large module → exceeds LLM context window → truncation → documentation quality degradation

CodeWiki's Dynamic Delegation:

Process a module
    ↓
Module complexity assessment
    ↓
    ├── Can be processed in a single pass → directly generate documentation
    │
    └── Exceeds capacity → spawn child agents
                   Sub-module 1 → Child Agent 1
                   Sub-module 2 → Child Agent 2
                   Sub-module 3 → Child Agent 3
                       ↓
                   After all sub-modules complete, parent agent aggregates

Each leaf agent has:

Stage 3: Hierarchical Aggregation (Bottom to Top)

Leaf module documentation (bottom layer, no dependencies)
        ↓
Parent module merges child module docs + generates architecture summary
        ↓
Top-level overview (overall architecture + system interaction diagram)
        ↓
Mermaid visualization generation:
    - Architecture diagram
    - Data flow diagram
    - Sequence diagram

Output structure:

./docs/
├── overview.md          ← Top-level architecture overview
├── module_A.md          ← Detailed documentation for each module
├── module_B.md
├── module_tree.json     ← Machine-readable module tree
├── metadata.json        ← Generation metadata
└── index.html           ← Generated with --github-pages option

Evaluation: CodeWikiBench

CodeWiki also contributed to its own evaluation problem — CodeWikiBench, a benchmark specifically for evaluating the quality of AI-generated code documentation.

Traditional text similarity metrics (BLEU/ROUGE) are unsuitable for evaluating documentation quality — a technically correct but verbose document might score high, while a precise, concise document might score low.

CodeWikiBench's evaluation approach:

1. Extract hierarchical evaluation rubrics (scoring criteria) from official documentation
   Generated using multiple models (Claude Sonnet 4, Gemini 2.5 Pro, Kimi K2)
   Semantic reliability 73.65%, structural reliability 70.84%

2. Multiple Judge Agents make binary judgments (pass/fail)
   Only judge at leaf nodes, avoiding ambiguous intermediate scores
   Judge models: Gemini 2.5 Flash, GPT OSS 120B, Kimi K2

3. Weighted scores aggregated upward from leaves
   With standard deviation confidence intervals

Key Results:

System Average Score
OpenDeepWiki (open-source) 47.13%
deepwiki-open (open-source) 50.05%
DeepWiki (closed-source, Cognition AI) 64.06%
CodeWiki 68.79%

CodeWiki has a clear advantage on Python/JavaScript/TypeScript (TypeScript +18.54%, Python +9.41%). Both sides perform generally poorly on C and C++, which the paper attributes to "language-specific parsing complexity" issues, with little relation to repository size.

Quick Start

Installation

git clone https://github.com/FSoft-AI4Code/CodeWiki.git
cd CodeWiki
pip install -e .

Generate Documentation

# Basic usage: generate documentation for the current directory
codewiki run . --output docs/

# Specify LLM provider
codewiki run . --provider openai --model gpt-4o

# Use Claude
codewiki run . --provider anthropic --model claude-opus-4-6

# Use Claude Code subscription (no API Key required)
codewiki run . --provider claude-code

# Generate GitHub Pages
codewiki run . --github-pages

# Incremental update (only regenerate modules changed since last run)
codewiki run . --update

Supported LLM Providers

Provider Method
OpenAI API Key
Anthropic Claude API Key
Azure OpenAI API Key
AWS Bedrock IAM
Atlas Cloud API Key
Claude Code Subscription, no API Key needed
Codex CLI Subscription, no API Key needed

Comparison with Similar Open-Source Projects

There are several projects worth knowing in this space:

deepwiki-open (AsyncFuncAI)

OpenDeepWiki (AIDotNet)

context-labs/autodoc

Summary of Solution Positioning

Project Stars Quality Deployment Tech Stack Suitable Scenario
CodeWiki 1.5k Highest (68.79%) CLI Python Large codebases, quality-focused
deepwiki-open 17.1k Medium (50.05%) Web UI Python/TS Quick deployment, web interface
OpenDeepWiki Not counted Medium (47.13%) Web UI C# .NET environment
autodoc Less maintained Early CLI Node.js Reference value

Project Links and Resources

Summary

CodeWiki's technical contribution is two-layered: a usable tool, plus an evaluation benchmark.

On the tool level, dynamic delegation solves a real engineering problem — large codebases cannot be stuffed into an LLM's context window in one go. Across tests ranging from 86K to 1.4 million lines of code, hierarchical recursive aggregation maintained documentation quality rather than degrading at truncation boundaries.

On the evaluation level, CodeWikiBench fills a gap: there was previously no scientific evaluation framework specifically for codebase documentation quality, and this work has value independent of the CodeWiki tool itself.

Limitations are real: performance on C and C++ lags behind Python/TypeScript; the star count (1.5k) is far lower than deepwiki-open (17.1k), indicating a gap in usability and community operations.

For scenarios requiring deep processing of large codebase documentation, CodeWiki's quality data is currently the most convincing among open-source solutions. For quick deployment and a web interface, deepwiki-open is the more convenient choice.

Explore PrimeSkills — a curated marketplace of AI Agents and skills, each verified through real enterprise workflows, stripping away hype and leaving what's truly useful.

Welcome to visit my personal homepage to discover more valuable insights and interesting products.