A Self-Growing Personal Knowledge Base with LLM Wiki, Obsidian, and Agent Tooling
This is Cang He's 576th original article!
Hello everyone, I'm Cang He.
Half-jokingly, I told a friend that if there were no AI, what I'd most want to be is a knowledge management blogger.
But honestly, methodologies like the Zettelkasten, PARA method, and bi-directional linking aren't that interesting to share.
And the most painful part is, you'll find that after a whole operation: organizing knowledge is more tiring than learning itself.
So, for a long time, I've been researching how to use AI for better knowledge management.
Recently, based on the LLM wiki knowledge base construction method and architecture made public by Karpathy, I've done some practical work.
Using WorkBuddy / Codex + LLM wiki + Obsidian, I built a self-growing personal knowledge base.
It can take clipped articles, AI conversations, and casually noted inspirations, and through WorkBuddy / Codex, continuously precipitate knowledge into an evolving knowledge brain.
This article is mainly divided into the following sections to break down the core construction ideas and practical steps of this system for everyone.
The content will be a bit long. You can like and save it, then throw it to your Agent to put into your own personal knowledge base system.
What is LLM wiki
To understand LLM wiki, you first need to figure out what problem it actually solves.
The current mainstream approaches, whether RAG systems, NotebookLM, or GPT's file uploads, are essentially the same process: you upload materials first, and when you ask a question, the model recalls relevant knowledge fragments and temporarily synthesizes an answer.
It sounds smart, but the problem is obvious.
Every time you ask a question, the model reassembles an answer from fragments, and it's gone after answering. Ask the same question a hundred times, and it reassembles it a hundred times; the knowledge itself has no accumulation.
The LLM wiki proposed by Karpathy can be understood as giving AI a long-term job: continuously maintaining a knowledge base.
Every time a piece of material is added, the Agent first checks the existing pages. Existing content is supplemented, new concepts get their own pages; when encountering different viewpoints, the source, time, and scope of application are kept together. This makes the divergence of knowledge traceable and prevents it from being quietly erased by a seemingly complete summary.
I think the key to "self-growing" lies right here: after the AI processes the material, the knowledge base must leave a change. It might add a new concept, supplement a connection, or expose a question that currently has no answer. Only by accumulating changes time after time can a truly personal knowledge system be formed.
To make this process controllable, the entire system is divided into three layers: the Raw layer stores original materials like articles, papers, and chat logs; the Wiki layer precipitates concepts, entities, and topics organized by AI; the Schema layer specifies how the AI should archive, update, cite, and handle conflicts.
Simply put, Raw preserves evidence, Wiki records understanding, and Schema sets the rules. The practical work that follows is about making these three layers truly operational.
Why Obsidian?
This system uses Obsidian, so why not directly use Notion, Ima, or NotebookLM?
First, Obsidian is completely local and data-sovereign. A Vault is just a regular folder on your computer, notes are saved as plain text Markdown, and it doesn't depend on a specific platform's page database.
Private materials like chat logs and meeting minutes can stay on your own device. Whether to sync and where to sync to is entirely up to you. To put it bluntly, the tools can change, but your knowledge doesn't have to move with them.
This file format is also very suitable for Agent work. WorkBuddy / Codex can directly read directories, create pages, modify links, and maintain indexes. Every change can be located to a specific file and can be tracked via Git.
Obsidian's bi-directional links, backlinks, and knowledge graph connect concepts, people, tools, and topics, helping me discover core nodes, orphan pages, and new associations.
Moreover, Obsidian's plugin ecosystem provides further expansion space, from templates, queries, and version management to publishing and visualization, all can be added as needed.
In the entire system, Obsidian simultaneously plays three roles: storage base, human-machine interface, and knowledge observation window;
WorkBuddy / Codex is responsible for execution, while Obsidian is responsible for saving, connecting, and presenting the results. For a personal knowledge base that needs long-term use, local sovereignty, open structure, traceable changes, and expandable capabilities are already sufficient reasons to choose it.
What can WorkBuddy / Codex do?
In this system, WorkBuddy / Codex serves as the Agent execution layer. They can directly read directories and Markdown files in the Obsidian Vault, call searches, scripts, and commands for batch processing, and work according to the rules agreed upon in AGENTS.md.
Simply put, Obsidian provides the workspace, LLM Wiki provides the knowledge structure, and WorkBuddy / Codex is responsible for making this structure truly operational.
When new material enters the Raw layer, the Agent first understands the content, identifies concepts, entities, and topics within it, and then cross-references them with the existing Wiki. Existing knowledge is supplemented, new content gets pages created, related knowledge is linked; when encountering different statements, it also preserves the source, time, and scope of application, and synchronously updates the index and change log. This entire process is incremental maintenance, without needing to repeatedly rebuild the entire knowledge base.
During the query and routine maintenance phase, the Agent is also responsible for retrieving relevant pages, organizing answers, tracing sources, and batch-checking for broken links, missing fields, and content that hasn't been updated for a long time.
It undertakes execution and orchestration in the system, while the human is responsible for setting rules, checking results, and handling key judgments. In this way, knowledge management gradually transforms from a large amount of manual organization into a continuous maintenance process of human-machine collaboration.
Practical Application
Next, let's enter the practical application phase. For the Agent choice here, you can use Codex, Claude Code, or WorkBuddy. For the model, it's recommended to choose one with a longer context length.
Regarding the model, if you're pursuing the ultimate cost-effectiveness, DeepSeek V4 Flash is currently a good choice, but unfortunately, it doesn't support multimodality.
I strongly advise against using GPT 5.6 Sol in Codex, because the token usage is really aggressive.
Of course, if using domestic models, you can also choose native multimodal models like Kimi K3, or models like Doubao-Seed-Evolving, which are optimized for Agents and long-range tasks and are dynamically and perpetually evolving.
In long-chain, multi-turn tasks, the model needs to continuously read documents and images, call tools, maintain context, and continuously adjust subsequent actions based on intermediate results. Therefore, context capacity, tool-calling stability, instruction following, and structured output capability all directly affect whether the entire process can run stably.
When actually selecting a model, it's recommended to run a complete task with your own real materials, focusing on observing whether context is lost, whether tool calls are stable, whether the output format drifts, and the time and cost of a single task.
Preparation
To build such a knowledge system, you need to prepare some tools and the environment in advance.
First is Obsidian. After downloading it for free from obsidian.md, create a new Vault:
Remember to keep your proxy on throughout the process, especially when accessing the plugin marketplace.
Here's a little easter egg: after downloading Obsidian, go to the plugin marketplace and download the WeSight plugin. This can also help you build this knowledge brain more smoothly.
If you're interested, you can look at the simpler and smoother building method later in the article.
After having Obsidian, you also need to prepare WorkBuddy or Codex. I won't detail how to download and use them here; you can also directly check the tutorials on our open-source blue book.
WorkBuddy: workbuddy.homes Codex: codexguide.ai
Of course, this current article will also be synchronized to the blue book.
LLM Wiki Three-Layer Architecture
The overall three-layer architecture of the LLM wiki theory can be customized according to your own needs, or you can reuse my template:
●●●
📦 my-wiki/├── AGENTS. md ← Schema specification layer: work instructions for AI (core)├── index. md ← Global index/site map (entry point for AI to locate knowledge, navigation directory)├── log. md ← Change log (append-only, clearly shows who changed what)│├── raw/ ← Raw material layer (read-only, AI must not modify)│ ├── articles/ ← Clipped articles, web pages│ ├── papers/ ← Papers, reports│ ├── books/ ← Books, highlights, notes│ ├── chats/ ← Valuable AI conversation records (backfilled via File Back)│ └── notes/ ← Inspiration fragments casually noted down│ └── meetings/ ← Meeting minutes transcriptions│└── wiki/ ← Wiki layer (fully maintained by AI) ├── sources/ ← Source summary pages: one page per raw material ├── concepts/ ← Concept pages: explaining methodologies, theories, patterns ├── entities/ ← Entity pages: people, companies, products, tools └── topics/ ← Topic overview pages: comprehensive comparative analysis of a field
How to Build
Through my personal practice, there are three methods to build this system, which I think are relatively friendly and simple enough for most people.
Below, I'll introduce them in order of difficulty, saving the simplest method for last.
Method 1: Direct Build Method
The first method is to directly have WorkBuddy or Codex build it based on the three-layer architecture template.
You can open the newly created Obsidian Vault in WorkBuddy and enter this directory:
Select 'Daily Office', then enter the following prompt:
●●●You are now the building and maintenance Agent for this personal knowledge base. Please build a long-term, incrementally maintainable LLM Wiki in the root directory of the current Obsidian Vault.
## Goal
Establish a three-layer structure of Raw, Wiki, and Schema:
- The Raw layer stores original materials, read-only, do not modify the originals.- The Wiki layer stores structured knowledge organized by AI, incrementally maintained by the Agent.- The Schema layer defines rules for archiving, updating, citing, and conflict handling through `AGENTS.md` in the root directory.
## Structure to Create
- `AGENTS.md`: Long-term work specification for the Agent- `index.md`: Global index and knowledge navigation entry- `log.md`: Change log, append new records only- `raw/articles/`: Article and web page clips- `raw/papers/`: Papers and reports- `raw/books/`: Books, highlights, and reading notes- `raw/chats/`: Valuable AI conversations- `raw/notes/`: Inspirations and casual notes- `raw/meetings/`: Meeting minutes and transcriptions- `wiki/sources/`: Source summary pages corresponding to raw materials- `wiki/concepts/`: Concept, theory, and method pages- `wiki/entities/`: People, company, product, and tool pages- `wiki/topics/`: Cross-source topic overview pages
## Rules AGENTS.md Must Include
1. Before processing new material, first search existing related pages in `wiki/` to decide whether to supplement old pages or create new ones.2. `raw/` is the source of truth; rewriting, deleting, or moving files within it is prohibited.3. For each piece of Raw material, create a corresponding source summary page in `wiki/sources/` and retain a link to the original file.4. Concepts, entities, and topics use independent pages, establishing relationships through Obsidian bi-directional links, avoiding piling all information into one note.5. All factual content must cite sources; content that cannot be confirmed must be explicitly marked as "To be verified".6. When new and old materials diverge, retain different statements along with their sources, time, and scope of application; do not directly overwrite old conclusions.7. Only update affected pages each time, and synchronously maintain related bi-directional links, `index.md`, and `log.md`.8. `log.md` uses an append-only method, recording date, material source, newly created pages, updated pages, and items requiring manual confirmation.9. Do not delete existing files without authorization; when a file with the same name already exists, first read and merge necessary rules, preserving the original content.10. When information is insufficient or a judgment might affect the overall structure, pause execution and ask me.
## Page Templates
Please create `_template.md` in `wiki/sources/`, `wiki/concepts/`, `wiki/entities/`, and `wiki/topics/` respectively. Templates should use Obsidian-compatible YAML properties, at least including: `title`, `type`, `aliases`, `tags`, `sources`, `created`, `updated`. The body should reserve areas for summary, core content, related pages, sources, and questions to be verified.
## Execution and Acceptance
Before executing, first check the current directory to confirm this is an Obsidian Vault; if unable to confirm, please ask me first. Then create missing directories and files, without overwriting existing content.
After completion, perform a self-check and report to me:
- Which directories and files were newly created- Which files were retained or merged because they already existed- The most important maintenance rules in `AGENTS.md`- Whether the current structure passed the check- Which directory the first test material should be placed into next
You can see the Agent will create the target files and directories according to the rules, and define AGENT.md
The Agent will automatically build the relevant structure as required.
At this point, when you open Obsidian, you can see the corresponding structure has been built:
Then, save an article to verify. Here, use a browser plugin called Obsidian Web Clipper, which can clip web articles into Obsidian with one click.
When choosing the actual directory, select the raw/articles directory we just created.
Then you can see in Obsidian that the article has been synchronized to the corresponding directory, which is very convenient:
Next, continue in WorkBuddy or Codex to build the wiki. You can refer to the following prompt:
●●●Please read the newly added articles in `raw/articles/` and strictly follow the rules in `AGENTS.md` to incrementally maintain the Wiki.
Before processing, first search existing pages: create a source summary for the article, extract concepts, entities, and topics from it; supplement existing content to original pages, create pages for new content, and retain source, time, and scope of application for differing viewpoints. Then establish relevant bi-directional links, and update `index.md` and `log.md`.
Do not modify the original files in `raw/`. After completion, tell me which pages were newly created or updated, and what content requires manual confirmation.
The agent will build entities, concepts, source summaries, topics, and other wiki pages according to AGENTS.md. Here you can see WorkBuddy has already created 9 new pages for us.
Going to Obsidian, you can also see the newly created wiki page information:
These Wiki pages can certainly be read directly, but their more important value is becoming the Agent's long-term retrieval layer and reasoning base. After structured organization, concepts, entities, topics, and source relationships originally scattered across articles become knowledge nodes that can be directly located, cross-referenced, and continuously updated.
When we ask questions again, the Agent can prioritize retrieving already precipitated Wiki pages, then follow bi-directional links back to related materials, quickly organizing well-founded, traceable answers. As more materials are processed, this reusable intermediate knowledge layer will continuously accumulate, making subsequent queries and maintenance increasingly efficient.
For example, I asked in WorkBuddy:
Based on the article just ingested and the existing Wiki, please answer three questions: What core viewpoints does the article propose? What are the relationships between the concepts, entities, and topics involved? Which conclusions can be transformed into actionable advice?
When answering, please cite the referenced Wiki pages and Raw original file paths; if there are differing viewpoints, information gaps, or content to be verified in the materials, please also list them separately.
You can see that the Agent first locates relevant knowledge nodes from the Wiki, then completes cross-referencing and synthesis along the bi-directional link relationships between pages, and traces key conclusions back to the corresponding Raw original texts; the entire answering process thus has higher retrieval efficiency, contextual consistency, and factual verifiability.
The above is Method 1, which is relatively crude and requires a very thorough understanding of prompts and engineering, and AGENTS.md needs constant adjustment to fit one's own knowledge base building needs.
Method 2: claude-obsidian plugin
Next, let's introduce Method 2. A skilled developer has already implemented a well-engineered open-source project based on the LLM wiki theory, called claude-obsidian.
It encapsulates common methods into plugins and Agent Skills, such as initializing the knowledge base, saving notes to the wiki, and automatic indexing and linking.
You no longer need to implement LLM wiki capabilities through prompts and custom instructions; you just need to use this project to build your own personal self-growing knowledge base.
This method is simpler than the previous one. You only need to install the project in WorkBuddy / Codex:
The current Vault will automatically create the structure. Just a simple ingest will sync new material to the wiki:
Use retrieve to quickly query results from the knowledge base:
You don't even need to explicitly specify retrieve; the Agent will hit the wiki in the knowledge base for fast retrieval:
Method 2 is a step further than Method 1: the project encapsulates initialization, ingestion, indexing, and retrieval into reusable Agent Skills, eliminating the work of repeatedly writing prompts and maintaining complex rules.
However, the main operations still occur within WorkBuddy / Codex. Users need to understand and call commands like ingest and retrieve; Obsidian mainly serves for file storage and result browsing, lacking intuitive visual feedback for task status, execution process, and knowledge changes. For users encountering this architecture for the first time, the barrier to entry still exists.
To truly integrate this knowledge base into Obsidian, the operation entry point also needs to enter the Vault. Thus, there is a third method:
Method 3: WeSight Knowledge Brain
The WeSight plugin integrates knowledge base initialization, material ingestion, Wiki updates, and intelligent retrieval into Obsidian itself, allowing users to complete the entire process within the familiar note-taking interface, while the backend Agent handles structural maintenance and continuous updates.
In this way, what the human sees is clear, visualized operations and results, while the Agent handles complex knowledge orchestration. The two ultimately form a complete human-machine collaboration loop.
Now, you just need to enable the 'Knowledge Brain' capability, and WeSight will automatically configure the environment and structure for your current Vault.
You can add the current note to the corresponding wiki of the Knowledge Brain with one click:
When chatting, select the 'Based on Knowledge Base' mode. WeSight will prioritize retrieving Wiki pages already precipitated in the Vault, and follow bi-directional links to locate related concepts, entities, and original sources, then organize answers based on this context. The entire retrieval, citation, and context assembly process is completed automatically, without needing to manually specify directories or call commands.
Moreover, you can save chat logs from Claude Code or Codex to the Knowledge Brain with one click, facilitating subsequent retrieval, association, and reuse, allowing valuable conversations to continue precipitating into long-term knowledge assets.
However, the WeSight Knowledge Brain feature is currently only open for a small number of members for internal testing. Everyone is welcome to experience it.
Why This Combination is "Self-Growing"
The "self-growing" nature of this combination comes from the continuous incremental updating of the knowledge structure. Whenever new material enters the Raw layer, the Agent searches the existing Wiki according to the Schema, supplements existing pages, creates new nodes, establishes bi-directional links, and records sources and viewpoint conflicts. Each processing leaves behind reusable structured results, and the knowledge base evolves continuously with the input.
Obsidian provides a stable local carrier and visual interface, WorkBuddy / Codex provides understanding, judgment, and orchestration capabilities, and WeSight further integrates ingestion, updating, and querying into the daily note-taking workflow. The human is responsible for inputting high-value materials and making key judgments, the Agent handles repetitive maintenance, and data, rules, and execution thus form a closed loop.
For material collection, on mobile you can use ima or other clipping tools for collection, or even directly use the WorkBuddy Mini Program to quickly collect articles and inspirations:
For collecting articles on a computer, you can directly use the Obsidian Web Clipper browser plugin for one-click clipping.
For other scattered materials, like Feishu documents, you can also use the Agent to quickly collect them into the knowledge base.
Final Thoughts
Looking back, the biggest change this system brings is transforming knowledge management from a one-time manual organization task into a continuously running maintenance mechanism. After articles, notes, conversations, and inspirations enter Raw, the Agent completes archiving, association, and updating according to rules, while Obsidian handles long-term preservation and presentation. Scattered materials thus gradually grow into a personal knowledge network.
Of course, self-growing doesn't mean full automation. Which materials are worth keeping, how rules are set, and whether key conclusions hold up still require human judgment; AI is better suited for repetitive, time-consuming, and structured maintenance work. Only when humans and Agents each handle what they are good at does knowledge management have a chance to persist long-term.
If you want to try it too, you don't need to build a perfect system from the start. First, create the three-layer directory, put in a piece of real material, let the Agent complete the first incremental update, and then continuously adjust the Schema based on actual use. As long as each input leaves behind reusable results, your knowledge base has already begun to grow.