Akashic: A Continuous Knowledge Loop for AI-Driven Development
一、引言
With AI increasingly participating in daily development, team knowledge is gradually scattered across code, documentation, and development conversations: code carries implementation details, documentation accumulates architecture and rules, and conversations retain the trade-offs, boundaries, and experiences behind problems. In actual development, we found that relying solely on manual effort makes it difficult to promptly capture code changes and conversation conclusions into the knowledge base: after code changes, related explanations may be omitted; conclusions confirmed during development may only stay in a single conversation. Switching tasks or colleagues means we still need to re-supply context and verify boundaries.
To reduce the repeated supplementation and verification during development, we launched the Akashic knowledge engineering practice: first, build an engineering knowledge base with a unified structure to provide reliable context for AI development tasks; when code changes, check whether related knowledge needs to be updated; after archiving development conversations, extract reusable experience from the dialogs, and write it back to the knowledge base after confirmation. This forms a continuously accumulating closed loop: the knowledge base supports development tasks, tasks generate code and conversations, and code and conversations then drive knowledge base updates, allowing the path one person has taken or the pitfalls they have encountered to become reusable experience for team members in subsequent tasks.
二、系统架构
The overall system consists of six modules: Akashic CLI, local development, engineering knowledge, session management, knowledge evolution, and conversation insights.
- Local base (Akashic CLI): The entry tool on the developer's local machine, uniformly managing configuration, Skills, and Git Hooks, responsible for scanning, filtering, and uploading development sessions from Codex and Claude Code.
- Local development layer: Developers use AI for solution design, coding, and troubleshooting. During the process, they read the corresponding part of the knowledge base based on task type, and produce two types of artifacts: development conversations and code changes.
- Engineering knowledge layer: Maintains a three-layer knowledge base structure in the Git repository: L1 entry, L2 documents and Skills, and L3 source code comments. The knowledge engineering Skill handles initialization and continuous write-back of coding knowledge for team members and their Agents to read on demand.
- Session management layer: Provides a unified session management API, archives raw conversations to OSS, forms team data assets, and provides a data source for conversation knowledge evolution and conversation insights.
- Knowledge evolution layer: Extracts reusable candidate knowledge from archived sessions. The Session Agent refines, the Audit Agent reviews and merges, and finally submits as an MR, which the Owner reviews before merging into the repository.
- Conversation insights layer: Computes metrics such as session rounds, tokens, and time span, and combines with Agent-extracted topics, summaries, and labels to form reports, helping teams identify knowledge gaps and recurring issues in development, and determine the AI collaboration capabilities to optimize next.
三、知识库结构
To enable continuous knowledge accumulation and updates, we first need to design the knowledge base structure and build a baseline version.
The core principle of our knowledge base is: Knowledge is uniformly maintained in the Git repository, based on source code, and the latest knowledge always follows the code. The knowledge in the repository includes global conventions, module rules, and local implementation details; they have different usage frequencies, scopes, and change frequencies. Putting all this into AGENTS.md would cause each task to load a large amount of irrelevant context and increase maintenance costs. Therefore, we divide knowledge into three layers based on scope and read timing: L1 (AGENTS.md) provides the entry and global routing; L2 (Skills + repository MD) carries knowledge reused per task; L3 is close to code evolution, explaining constraints in specific implementations. This allows the Agent to delve deeper based on the task.
L2 structure example:
knowledge/
├── README.md # L2 master index
├── docs/
│ ├── 01-Positioning and Goals.md # Basic positioning
│ ├── 02-Overall Architecture.md # Link architecture
│ ├── 03-Module Index.md # Source navigation
│ ├── 04-Core Concepts and Terms.md # Terminology
│ ├── 05-Key Data Flows.md # Request and field flows
│ ├── 06-Configuration and Extension Points.md # Config and extensions
│ ├── 07-Dependencies.md # Cross-repo dependencies
│ ├── 08-Development Guide.md # Development and verification
│ ├── modules/ # Module knowledge
│ │ ├── foundation/ # Common foundation capabilities
│ │ ├── init/ # Request initialization
│ │ ├── recall/ # Multi-channel recall
│ │ ├── merge/ # Candidate fusion and dedup
│ │ ├── berank/ # Coarse ranking
│ │ ├── aerank/ # Fine ranking
│ │ ├── rerank/ # Re-ranking
│ │ └── ...
│ └── topics/ # Technical topics and business domains
│ ├── ab-params/ # Experiment parameters
│ ├── coldstart/ # Cold start
│ ├── ad-fusion/ # Ad fusion
│ └── ...
└── skills/ # Task-oriented operational procedures
├── recall-channel-dev/ # Recall channel development
│ ├── SKILL.md
│ └── script/ # Companion execution script
├── metric-anomaly-debug/ # Metric anomaly troubleshooting
│ ├── SKILL.md
│ └── script/
├── feature-field-add/ # Feature field addition
│ ├── SKILL.md
│ └── script/
└── ...
When actually executing tasks, the Agent will progressively load knowledge base information based on task type. For example, when adding a new recall channel, the Agent first finds the recall module entry from AGENTS.md, reads the module documentation, understands how the recall results enter the subsequent pipeline; then uses the "recall-channel-dev" Skill to complete integration and verification according to conventions. When experimental parameters are involved, it additionally reads the Ab-Params topic; after locating the specific implementation, it confirms local constraints through source code comments. In this way, module documentation provides background, Skills provide execution methods, and source code and comments provide implementation basis.
四、知识库初始化与代码更新同步
After determining the knowledge base structure, the next step is to establish and continuously maintain it. To unify the initialization method and integrate knowledge updates into the daily development workflow, we developed the Akashic-Knowledge Skill, which covers two processes: establishing a full-repo knowledge baseline when first integrating a repository, and checking and updating related knowledge based on the current code changes during daily commits.
- First integration initialization: In the repository directory, invoke the Akashic-Knowledge Skill. The Agent combines existing documentation, source code, configuration, and test cases to verify engineering facts, establish the L1 knowledge entry, L2 module and topic documentation, and add necessary L3 source code comments. After confirmation and validation, it is written to the repository, recording the initialization state as the starting point for subsequent incremental synchronization.
- Code commit synchronization: When the Agent is about to submit this round of code changes, a script locates potentially affected documents based on the changed files and the mapping relationship in the knowledge base, and provides diff evidence; the Agent then combines with source code to determine whether module responsibilities, configuration rules, or implementation boundaries have changed. When updates are needed, it generates corresponding knowledge modifications, which are confirmed and validated and committed together with the business code; when no update is needed, it only records the judgment result. This keeps knowledge in sync with code at all times without manual proactive maintenance.
五、对话知识回写
The previous chapter described how to synchronize related knowledge when code changes, but much knowledge is not necessarily produced alongside code modifications; it is hidden in development conversations. We conducted an in-depth analysis of the team's daily development conversations and found that there are many instances of manual supplementation and correction. For example, a developer may confirm the effective scope of a configuration during troubleshooting, or correct the Agent's judgment on field semantics and effective links. If these conclusions only remain in historical sessions, subsequent tasks still have to re-explain and verify. Therefore, we extract candidate knowledge from the team conversations archived in OSS, combine with conversation evidence, verify against source code and existing knowledge as needed, and after review, generate an MR for the Owner to review.
The division of labor in conversation knowledge write-back is: the model is responsible for knowledge refinement and review, with the Agent's permissions limited to reading, retrieval, and editing; the program handles interaction with external systems (OSS, MySQL, GitLab), process orchestration, and change validation; the Owner is responsible for final review. The specific process is as follows:
Refinement and review use the same OSS session snapshot, as well as fixed versions of source code and existing knowledge, to avoid inconsistencies between before and after. Each session generates candidate modifications in an independent workspace.
Knowledge precipitation is divided into two phases: refinement and review. The Session Agent reads the original conversation, uses knowledge base routing to view existing knowledge on demand, refines conclusions and provides evidence; the Audit Agent then checks whether the evidence is sufficient, whether it duplicates or conflicts with existing knowledge, and ultimately gives a decision on inclusion or exclusion along with reasons. If there is no reliable, reusable conclusion, the knowledge base is not modified.
Knowledge gaps discovered from conversations can be classified into different layers of the knowledge base. Typical correspondences are as follows:
六、本地能力基座
The two knowledge write-back paths established by Akashic knowledge engineering both depend on the capabilities of the member's local machine: when code changes, the knowledge engineering Skill needs to be invoked; before extracting knowledge from conversations, the development session must first be archived. If installation, upgrade, and upload all depend on manual execution by members, problems such as inconsistent tool versions, outdated Skills, and forgotten session uploads can occur.
Therefore, we built the Akashic CLI to uniformly manage tool installation, updates, configuration synchronization, and session collection. Members continue to use their original AI coding tools and Git commit workflow; related capabilities run automatically with daily operations.
The entire onboarding process has three phases: first integration, daily trigger, and centralized update.
During first integration, the Agent completes the installation automatically. The repository's AGENTS.md defines the installation check and execution method. The Agent checks the local environment during daily tasks and executes a unified installation script when the CLI is missing. The script downloads a precompiled program, completes configuration initialization, Skill synchronization, and Git Hook installation, and triggers the first session scan. Members do not need to separately install the CLI, copy Skills, or configure reporting commands.
During daily development, Git commits trigger background maintenance. After a Git Commit in a repository within the integration scope, the Post-Commit Hook starts a background task to check tool version and configuration updates, and scan qualified Claude Code and Codex sessions for upload, without affecting code commits.
The CLI uses local state records to control duplicate reporting and failure retries. The same session is not reported again after a successful upload on the same day. Recently active sessions can form new snapshots within a configurable window (default three days), and the latest snapshot is selected for subsequent knowledge refinement. A single file is retried up to three times; if it still fails, the state is recorded and left for the next trigger to re-report.
After capability adjustments, unified distribution takes effect automatically. CLI version, Skills, repository whitelist, and session collection time window are configured centrally. After configuration is published, the local machine automatically fetches updates at the next daily trigger.
In this way, the distribution of knowledge engineering capabilities and session collection are integrated into existing development actions, making the solution non-intrusive to the team members' daily development workflow. Team members do not need to perceive additional operational processes.
七、对话洞察
To further mine the team's experience and problems in AI applications from conversations and provide a basis for the team's AI engineering construction, we also built a conversation insights pipeline: after archiving, a program counts metrics such as user rounds, duration, and tokens, and a read-only Agent summarizes topics, summaries, and labels session types and multiple signals such as "supplement knowledge", "adjust direction", "request redo", and "confirm effective". The report supports filtering by time, repository, and tags. The team can go from aggregated data back to specific sessions to verify the context when a problem occurred.
The conversation insights report helps teams identify problems in AI application, for example:
- When rounds increase and "supplement knowledge" increases: may be due to business background or module explanations repeatedly needing to be supplemented by developers, and we need to check whether relevant knowledge base links cover this.
- When rounds increase and "request redo" increases: may indicate issues with task understanding, execution paths, or result validation, and we need to drill down into relevant conversations to judge, and if necessary, build relevant tools to solve them.
It also helps teams review practice effectiveness, for example: after introducing knowledge engineering in a repository, as the knowledge loop writes updates, the rounds and correction labels for similar tasks effectively decrease.
八、结语
In the past, knowledge maintenance often had to wait for developers to finish tasks and then manually organize. Code changes continuously, and conversation conclusions and solution trade-offs are constantly generated, but knowledge base documents and Skill capabilities are difficult for humans to keep up with in time. Akashic integrates knowledge base initialization, code change checks, session collection, and candidate knowledge extraction into the development workflow, so that content that needs maintenance is discovered and updated along with the development process, no longer relying on post-hoc supplementation.
In the current phase, Agents are responsible for refining and reviewing candidate content, code and conversations provide verification basis, and Owners review before merging. As evidence verification, conflict detection, and change validation gradually improve, we hope to automate updates with sufficient evidence and manageable risk, leaving manual judgment for cases with insufficient evidence or involving business trade-offs.
往期回顾
1.Stress Testing AI Agent Governance and Evaluation Tuning Practice
3.Jev in Practice: Introducing a Lightweight Decision Layer for AI Workflows, 10x Speedup | Dewu Tech
文 /知瑜
Follow Dewu Tech, technical articles every Thursday
If you find the article helpful, feel free to comment, share, and like.
Unauthorized reproduction without Dewu Tech's permission is prohibited, and legal liability will be pursued.