Recursive Self-Improvement: How AI Systems Learn to Improve Themselves Across Rounds
Recently I've been working on DocFlow, a full-stack hands-on project combining Agent, RAG, and knowledge graphs, tested to support real-time collaboration for a hundred people, with support for Word (DOCX) and PDF import. The Agent can search materials, rewrite documents, and operate on spreadsheets. Under the hood it implements a main Agent coordinating, sub-agents verifying in parallel and reviewing results, as well as automatic context compression, long-term and short-term memory, and task recovery. RAG covers parent-child chunking, hybrid BM25 + vector retrieval, RRF fusion, reranking, and source citation tracing, suitable for those who want to dig into source code and build AI applications hands-on.
I've also curated 80 articles on Agent development, 72 on RAG, and 70 on knowledge graphs, 222 total learning resources, from basic principles to engineering practice. You can learn while reading the source code. If you're interested, add my WeChat yunmz777 to learn about the project and accompanying materials, and to discuss AI development together.
When we ask an AI to complete a task, it finds errors and keeps modifying until the result meets the requirements. This process is already common. But after the task ends, two other questions arise: did this correction change the system, and can the next improvement use the capability accumulated this time?
RSI studies exactly this kind of self-improvement that can persist. Its full name is Recursive Self-Improvement, in Chinese 递归自我改进. The core process is to let AI use experience and feedback to modify itself, retain verified changes, and let the updated system participate in the next round of improvement.
To understand RSI, you need to track two objects continuously: the system's ability to handle tasks, and the system's ability to generate improvements.
They can improve together, or only one of them may improve. Once this relationship is clear, model training, tool modification, experience accumulation, and validation can all be understood along a single main thread.
RSI changes must stay with the system itself
Suppose an AI writes a program with errors, and it fixes the program based on test results. This change happens on the deliverable. If the next time it handles a similar task, it still uses exactly the same method, you cannot conclude from this single fix that the system has learned to improve itself.
Self-improvement is concerned with a different type of change: the system modifies its own tools, processing strategies, reusable experience, or model parameters, and continues to use them in subsequent runs.
For example, a Coding Agent often breaks files because it replaces ranges inaccurately. It can only fix the code broken this time, or it can further modify its own editing tool to add location checks and pre/post-modification verification, so that subsequent editing processes change.
The latter modification acts on the system itself and has the potential to persist across rounds.
The Last AI Built by Humans includes model parameters, Agent Harness, and improvement strategies in the scope of self-change. It focuses on where changes are saved, what successor systems inherit, and how inheritance affects subsequent improvements. Therefore, self does not only refer to model weights.
However, persistent storage does not equal effective learning. A log that is never read again only provides historical records; an experience, even if read, must be able to influence specific decisions to change system behavior.
Therefore, when judging self-improvement, I first identify the object that actually changes. Only by clarifying how it is saved and reused can we know whether the improvement crosses the current task.
Recursion lets the previous round's capabilities participate in the next round's improvement
Persistent modification explains self-improvement, but recursion requires one more step forward.
We can imagine a fixed optimization program that continuously modifies an Agent and keeps the better-performing versions. The Agent's task performance may keep improving, but the method responsible for finding modifications never changes. In this case, what we observe is mainly a fixed optimization process.
RSI further focuses on whether the updated capabilities can return to the improvement process and influence how the next round finds problems, proposes candidates, judges effects, or retains experience.
This return relationship can be established in different ways:
- After tools are improved, the system can more easily modify its own implementation.
- After problem-diagnosis capability improves, the next round can locate failure causes more accurately.
- When past experimental experience is retained, the system reduces repeated attempts at ineffective solutions.
- After the model is trained, it can generate higher-quality improvement candidates.
This does not require rewriting the source code of the improver every round. Model capabilities, tools, and experience changing can also alter the actual decisions made by the same program.
Harness² provides a concrete example: the system adjusts how the current task is executed, then organizes which modifications in this adjustment were effective and which showed no observable effect, for later improvement processes to reference. New tasks start from a base Harness, but the process responsible for proposing modifications can use existing experience, so the previous round's results enter the decision of how to improve next.
Recursion describes this feedback relationship, not how many times a program runs, nor whether a function must call itself.
Executing tasks and finding improvements can be done by the same model, or by different components. The key is whether the system's inherited changes actually participate in subsequent improvements.
Systems can change parameters, or change how capabilities are used
Once the recursion relationship is clear, we can judge what a specific approach is actually changing.
Model parameters determine part of the reasoning and decision-making capabilities. The program that organizes the model's execution affects how these capabilities are invoked. Tool interfaces, information retrieval methods, skill selection, and error handling can all become targets of modification.
| Modification target | What specifically changes | Possible subsequent impact |
|---|---|---|
| Model parameters | Reasoning, decision-making, and tool-use behavior | More accurate task execution or modification proposals |
| Prompts and strategies | Work order, selection conditions, and inspection requirements | Adjust how the system handles problems |
| Tools and code | Editing, searching, calculation, and verification capabilities | Improve action accuracy and execution efficiency |
| Experience and skills | Reusable methods, applicable conditions, and failure lessons | Provide existing foundations for subsequent decisions |
| Improvement methods | Problem diagnosis, candidate generation, and experiment selection | Change how the next round seeks effective modifications |
These targets can be combined, but they do not automatically yield benefits just because they are stored. We still need to inspect actual behavior and results.
Take tools and code as an example. Darwin Gödel Machine lets a Coding Agent modify its own implementation, then evaluates the new version through coding tasks. The modification scope includes editing tools and workflows. It keeps multiple historical versions to continue exploring, while the base model weights remain frozen, showing that system-level self-modification can occur outside the model.
Parameter updates require training. The system can extract experience from execution, filter effective trajectories, form training material, and then get a new model through supervised training or reinforcement learning. If the updated model continues to participate in experience generation and subsequent improvements, a recursive relationship forms that needs further analysis.
Here is a practical distinction: training determines how parameters change, RSI focuses on how these changes enter the complete improvement process.
Similarly, memory determines how information is retained, tools determine how actions execute. All can become components of RSI, but no single component alone suffices to show the full recursive relationship is established.
One round of self-improvement goes through proposal, validation, and retention
A round of RSI usually starts from task results or environmental feedback. The system needs to convert this feedback into specific problems, then propose modifications that can be checked.
Suppose tool calls frequently fail. The cause could be unclear tool descriptions, incorrect parameter generation, or changed external environment. If we keep adding prompts without locating the cause, the system may store more rules without solving the actual problem.
Therefore, proposals need testable hypotheses, such as: after changing the tool's return structure, are key fields easier to use correctly? Next, generate candidate versions, execute them against the original under comparable conditions, and then decide whether to retain.
Validation must at least answer whether the target problem improved, whether original capabilities regressed, and how much extra cost the benefit requires. Candidates that fail to run, show regression, or lack evidence should have different outcomes. We should not treat them all as successful modifications just to keep the process going.
The retention stage also needs to save the scope of applicability of a modification. A change may only work for certain types of tasks. If applied indiscriminately to all tasks, it can turn local gains into global regression.
At the same time, the system needs to retain rejected approaches and their reasons. Knowing which directions have been tried and under what conditions they had no effect helps the next round reduce repeated exploration.
Improvement feedback and final acceptance must not be mixed
Execution records can help the system propose modifications, but records alone are not necessarily enough to judge whether results are correct.
For example, an Agent's final output saying the task is complete only indicates that it made that judgment; a tool call without errors only indicates that the call process met certain conditions. Whether the final result meets the task requirements still needs corresponding verification evidence.
For tasks that can be objectively verified, we can use tests, computed results, or environment states. For tasks with no single answer, we may need source verification, human evaluation, and multiple signals together.
HarnessPro takes the approach of generating tests based on task information, fixing them during evolution, using them to compare candidate configurations' pass rates and execution costs, and then checking final effects through independent evaluation. This design gives candidates a relatively consistent comparison basis, but generated tests may still miss requirements or misunderstand the problem.
Therefore, passing internal validation only shows that the candidate satisfies the current checks; it cannot be directly expanded to mean the candidate is more reliable in all situations.
If the system is allowed to improve its own internal evaluation methods, we also need to separately check whether the evaluator becomes more accurate. When modifying answers and modifying acceptance criteria happen at the same time, score changes may lose their original meaning.
Better at tasks and better at improving itself need separate validation
A round of modification increasing task performance shows that the new version performs better under corresponding conditions. To further judge effective recursion, we also need to check its ability to produce subsequent improvements.
I separate these two validations.
Validation of execution capability: compare new and old systems on tasks that were not part of optimization, checking accuracy, completion, old capabilities, and resource consumption.
Validation of improvement capability: let the original improvement method and the updated method start from comparable system states, search for modifications under similar resources, and then check the successor versions they produce.
The latter validation focuses on whether problem localization is more accurate, whether there are more effective candidates, whether ineffective attempts decrease, and whether the found modifications transfer to subsequent tasks.
If the new version simply tried many more times, or used a stronger external model, these factors need to be recorded separately and cannot all be attributed to the recursion mechanism.
This way, we can distinguish whether the system only completed more attempts, or learned more effective improvement methods from previous attempts.
Structural recursion can be confirmed by process; effective recursion requires such comparative evidence.
Self-improvement can accumulate gains, but also encounters bottlenecks
RSI does not guarantee that every round gets better. Candidate generation, feedback quality, and validation capability can all limit it.
If the system cannot judge where the problem is, it may repeatedly modify around the wrong cause. If useful tools or capabilities are missing, it may keep generating similar candidates. If validation coverage is insufficient, it may accept changes that appear effective but actually cause regression.
A simple example: a test only checks whether a file exists, not its content. Even if the system gets better and better at passing this test, it may not get better at completing the real task.
Improvement is also affected by resources and the task itself. As easy problems diminish, finding the next effective modification may require more experiments, and validating complex tasks may become a major cost.
Therefore, effective recursion does not require every round's gain to be larger, nor can it directly imply infinite growth.
What we need to observe is whether, under clear conditions and resources, inherited capabilities and experience continue to help the system find effective modifications, and honestly record cases where gains weaken, regress, or cannot be confirmed.
Summary
RSI connects experience, modification, validation, and inheritance into a continuous process. Its modification targets can be model parameters, or tools, strategies, and experience. Recursion happens when these changes re-enter subsequent improvements.
An improvement in task performance shows the execution capability changed; the system being able to find subsequent improvements more accurately and efficiently provides further evidence of effective recursion.
When understanding RSI, the relationship to see most clearly is: what exactly the previous round left behind, how it affects the next round, and whether that influence stands up to validation.