Artificial Intelligence · Open Source · News

CodeWiki Uses Recursive Agents to Generate Docs for Million-Line Codebases

By 冬奇Lab · Jul 30, 2026

Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Repository-level documentation has been a manual, neglected chore for decades. A system that can produce architecturally coherent docs for million-line codebases — and back its quality claims with a peer-reviewed benchmark — changes the calculus for teams deciding whether documentation is worth the effort.

Summary

CodeWiki, from FPT Software's AI4Code team, tackles repository-level documentation with a three-stage pipeline: Tree-sitter AST parsing builds a dependency graph, topological sorting determines processing order, and a recursive multi-agent system generates docs from the bottom up. When a module exceeds context limits, the system spawns child agents that document sub-modules independently before a parent agent aggregates the results. Output includes Markdown files with Mermaid architecture, data-flow, and sequence diagrams.

The project ships with CodeWikiBench, a benchmark that evaluates documentation quality using multiple judge models making binary pass/fail decisions on leaf-node criteria, avoiding the pitfalls of BLEU/ROUGE for this task. CodeWiki scores 68.79%, ahead of Cognition AI's closed-source DeepWiki (64.06%) and the open-source deepwiki-open (50.05%).

An incremental update mode regenerates only changed modules, and the CLI supports OpenAI, Anthropic, AWS Bedrock, and subscription-based providers like Claude Code that require no API key. The main limitation is weaker performance on C and C++ compared to Python, JavaScript, and TypeScript.

Takeaways

— CodeWiki parses source files with Tree-sitter across 9 languages, normalizes cross-file imports into a depends_on directed graph, and topologically sorts modules so dependencies are documented before the code that uses them.

— Dynamic delegation lets an agent assess module complexity and spawn child agents when a module exceeds a single LLM's context window; each child has access to source code, the global module tree, dependency traversal tools, and a registry to avoid duplicate generation.

— The three-stage pipeline produces Markdown docs with Mermaid architecture, data-flow, and sequence diagrams, plus a machine-readable module tree and metadata file.

— CodeWikiBench evaluates documentation quality by extracting rubrics from official docs, then using multiple judge models for binary pass/fail decisions at leaf nodes, with weighted scores aggregated upward and confidence intervals reported.

— CodeWiki scored 68.79% on the benchmark, beating DeepWiki (64.06%), deepwiki-open (50.05%), and OpenDeepWiki (47.13%), with the largest margins on TypeScript (+18.54%) and Python (+9.41%).

— Performance on C and C++ is weaker, attributed to language-specific parsing complexity rather than codebase size.

— The --update flag enables incremental regeneration, only processing modules that changed since the last run.

— Supported LLM backends include OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Atlas Cloud, Claude Code, and Codex CLI — the last two require only a subscription, no API key.

Conclusions

The gap between CodeWiki's benchmark score (68.79%) and its star count (1.5k) versus deepwiki-open's score (50.05%) and star count (17.1k) suggests that in developer tools, deployment ease and community momentum currently outweigh raw quality in adoption.

CodeWikiBench's design — binary leaf-node judgments from multiple models with weighted upward aggregation — sidesteps the well-known problem that BLEU/ROUGE reward verbosity over precision, and could serve as a template for evaluating other structured-generation tasks.

The recursive agent architecture is a practical answer to the context-window problem that doesn't require waiting for larger context windows; it trades latency and API cost for completeness, which is the right trade for a batch documentation task.

ACL 2026 acceptance gives the approach academic credibility, but the real signal for production use is the incremental update feature — documentation that rots slower than the codebase is what teams actually need.

Concepts & terms

Dynamic Delegation

CodeWiki's mechanism where a parent agent evaluates a module's complexity and, if it exceeds a single LLM's context capacity, spawns child agents to document sub-modules independently. The parent then aggregates their output. This avoids truncation-based quality loss on large modules.

CodeWikiBench

A benchmark for evaluating AI-generated codebase documentation. It extracts hierarchical rubrics from official docs, uses multiple judge models to make binary pass/fail decisions at leaf nodes, and aggregates weighted scores upward with confidence intervals — avoiding the verbosity bias of BLEU/ROUGE.

Topological sorting (in CodeWiki)

After building a depends-on directed graph from cross-file imports, CodeWiki topologically sorts modules so that zero-indegree nodes (leaf modules with no internal dependencies) are processed first. This ensures a module's dependencies are documented before the module itself.

Source: juejin.cn ↗ Google Translate ↗ Backup ↗