KotlinLLM Turns LLM Answers into Permanent Source Code at Runtime
The library challenges the default assumption that LLMs must be called repeatedly for every input. By persisting generated logic as version-controlled source code, it cuts token costs, eliminates latency for known scenarios, and makes LLM-authored code auditable — a model that fits regulated or cost-sensitive production environments.
JetBrains open-sourced KotlinLLM, a library that shifts LLM usage from a per-request service to a one-time code evolution event. When a program encounters unfamiliar data, a Koog agent generates a guard expression and a case handler in pure Kotlin. The plugin compiles the new code and hot-swaps it into the running JVM via JDI, so subsequent matching inputs execute the generated bytecode directly without any model call.
The system enforces strict guard logic: each guard must act as a scenario classifier, not a loose validity check, to prevent misrouting future inputs. Generated code is saved as ordinary Kotlin files that can be reviewed, tested, and committed to version control. A companion `mockLlm<T>()` entry point builds stateful interface fakes that remember call history, moving beyond fixed-return mocks.
In a GitHub Issue Radar demo processing over 30,000 issues across 20 repositories, the approach achieved a 0.89 recall rate for identifying beginner-friendly tasks, with compilation and hot-replacement accounting for roughly 1% of total time.
KotlinLLM inverts the dominant AI coding paradigm: instead of generating code in a chat window and pasting it into a project, the model writes and deploys code autonomously while the program is running, under strict structural constraints.
The guard-as-classifier requirement is the system's real safety mechanism. A guard that is too broad silently routes wrong inputs into a handler, producing garbage results with no error — a failure mode that is hard to detect in traditional prompt-based generation.
Hot-replacing bytecode in a running JVM for AI-generated logic is a sharp departure from the common pattern of restarting or redeploying. It suggests a future where applications self-extend without downtime, though the current IntelliJ plugin dependency limits production readiness.
The 12-call tool budget for the Koog agent, if enforced, would force the model to make decisions with incomplete information — a deliberate trade-off that prioritizes speed over exhaustive analysis and mirrors real-world constraints on agentic systems.