01 | What Is an Agent Harness? Why a Bare LLM Can't Run on Its Own

Author: 浪遏 Tags: AI Programming

This is the first article in the "Full-Stack Agent Development" series. The entire series uses catbuddy (a personal project: a local-first Agent, approximately 36,000 lines of TypeScript, built on a monorepo to construct desktop, web, and server-side Gateway) to dissect, from simple to complex, the engineering system behind an AI that can "get things done."

This first article clarifies the most fundamental question: the model is already so powerful, so why can't you just call the API directly to build a decent Agent? What exactly is missing in the middle? Finally, it gives you a panoramic map, so that when any detail is dissected later, you can place it in the overall picture.

A Contrast That Puzzled Me for a Long Time

The first time I used Claude Code to modify code, I was somewhat shocked.

I said, "Migrate this component's state management from useState to useReducer," and it actually read the file, understood the existing structure, modified the code, and conveniently updated the places that referenced it. The whole process involved it reading, writing, and verifying on its own, running for seven or eight rounds continuously.

Then I went back to my own project, using the same company's model and the same API Key, wanting to replicate this experience. What was the result? I called the API once, and it returned a large block of text saying "You could do this..." I pasted its answer back, and it returned another block. It knew what to do, but it could do nothing—it couldn't read my files, couldn't run my commands, and couldn't remember what we were talking about three sentences ago.

The same model, so why such a huge difference?

The answer is: Claude Code is not just "calling the model." A layer of something is wrapped around the model, and it is precisely this layer that transforms a "chatty model" into an "agent that gets things done." The industry calls this layer the Harness.

1. The LLM API You Call Is Essentially a Stateless Text Completion Function

To understand what the harness supplements, you must first see clearly what a "bare model" actually is.

Stripping away all packaging, the essence of a single LLM API call is a pure function: output text = LLM(input text). You give it a piece of text (system prompt, conversation history, your question), and based on the probability distribution learned during training, it spits out the most likely next piece of text. That's it. This function has three "inherent disabilities," which are precisely the root cause of its inability to work independently:

Disability One: It has no memory. This function is stateless; between this call and the last, it has no "recollection." So-called "multi-turn dialogue" is entirely an illusion—it's actually because you re-feed all previous conversations into the input each time, making it "appear" to remember. If you don't feed it, it's an amnesiac.

Disability Two: It can't take action. Its output is always just text. It can say in text "I want to read src/index.ts," but it has no hands to actually read that file. It can describe actions but cannot execute them.

Disability Three: It cannot self-advance. You ask one question, it answers one, and then this call ends. It won't proactively say, "I'll read a file first, then decide the next step after seeing it"—because it cannot perform the action of reading a file (Disability Two), and after one call ends, it "dies" (Disability One).

What does a real programming task look like? "Help me fix this bug"—this requires: first look at the error, then read the relevant files, locate the problem, modify the code, run tests, see the results, and possibly modify another round. This is a process that requires repeated action, observation, and re-decision. A bare LLM can only do the small "decision" step within that; it cannot do the rest of the action, observation, or advancement.

2. From "Completing a Paragraph" to "Completing a Task"

So what to do? Patch the three disabilities one by one:

• It has no memory → We store the history externally and feed it to the model on each call;
• It can't take action → We give it a set of tools. When it says in text "I want to call read_file," we actually go read it and feed the result back;
• It cannot self-advance → We wrap it in a loop: call the model once, see what it wants to do, do it, give the result back, call it again, until it says "I'm done."

Implement these three things, and you get the most basic Agent, whose skeleton looks like this:

See that ring looping from Exec back to Call? That is the soul of the entire Agent—the industry calls it the Agent Loop. It transforms "one-time text completion" into "continuous action-observation-re-decision." The model is responsible for thinking, and the loop is responsible for allowing the thinking to land and continue.

This entire engineering system that wraps the "bare model" and enables it to continuously complete tasks is the Harness.

3. A One-Sentence Definition: The Harness Is the Chassis Outside the Engine

If a definition is necessary:

The Harness is the entire engineering system wrapped around a large model that transforms "one-time text completion" into the "continuous task-completing behavior of an intelligent agent."

My favorite analogy is a car:

Car	AI Agent
Engine	Large Model (provides power/intelligence)
Chassis, Transmission	Agent Loop (transmits power into forward motion)
Steering Wheel, Gas, Brake	Tool System, Interrupt Control (allows it to operate, to stop)
Dashboard, Rearview Mirror	Context Engineering (determines what the driver can see)
Dashcam	Memory System (remembers the road traveled)
Airbags, ABS	Reliability Engineering (prevents falling apart in an accident)

No matter how strong the engine, without a chassis, steering wheel, and brakes, it's just a machine roaring in place, going nowhere. The model determines the upper limit of the agent; the harness determines whether it can actually drive on the road.

4. A Piece of Code to See the Difference

Just talking concepts is a bit abstract; let's look at code. First, the "bare call"—this is what most people write when they first encounter the LLM API:

// Bare call: one question, one answer, and that's it.
const res = await llm.chat({
  messages: [{ role: "user", content: "Fix the type errors in src/app.ts" }],
});
console.log(res.content); // → "You can open app.ts, find line X..." (And then? Nothing.)

It only tells you what to do. Now look at it after being wrapped in a harness (extremely simplified, but the skeleton is exactly this):

// Harness: Loop + Tools, until the task is complete
const history = [{ role: "user", content: "Fix the type errors in src/app.ts" }];
while (true) {
  const res = await llm.chat({ messages: history, tools: TOOLS }); // Carrying tool descriptions
  history.push(res.message);
  if (!res.toolCalls) break;                        // Model says "done" → exit loop
  for (const call of res.toolCalls) {
    const result = await runTool(call);             // ⭐ Actually reads files / modifies code / runs commands
    history.push({ role: "tool", content: result });// Feeds the result back, continues to the next round
  }
}

The difference lies in that while loop and runTool. With just over ten lines added, the model goes from "armchair strategist" to "actually making changes." Of course, a real harness is far more than these ten lines—catbuddy's core loop, plus various governance logic, is over a thousand lines. But the core is exactly this shape. In the next article, we will write this core by hand and get it running.

5. Case Project: A Harness Where "Code Doesn't Leave Local"

Why does this series use catbuddy as the case study instead of directly discussing Claude Code? Because it has a very hard constraint that forced a set of interesting design trade-offs.

catbuddy's origin is a real scenario: someone was banned by their company's security department from using all cloud AI tools because the code contained business logic—they wanted to use AI-assisted programming, but the source code could not leave the local machine. The tools on the market fall into three categories, none of which simultaneously satisfy "strongest reasoning + code doesn't leave the machine":

Type	Representative	Problem
Pure Cloud	Web-based assistants	Code must be uploaded, compliance fails
IDE Plugin	Copilot/Cursor	Reasoning servers are still in the cloud
Pure Local Model	Ollama running small models	Model too weak, can't handle complex tasks

catbuddy's choice is a fourth path: let the strongest cloud model do the reasoning, but the Agent process runs locally. You use your own API Key to connect directly to Anthropic/OpenAI, but actions like reading files, modifying code, and running commands all happen on your machine; the source code never passes through any third-party server. This "local-first" constraint will repeatedly surface in many later designs—why the Gateway is just a "post office that doesn't touch files," why storage is local JSONL files, why the workspace needs sandbox isolation—remember it, and many trade-offs will become logical.

6. Panoramic Map: The Six Organs of the Harness

After all this talk, it's time to give you a global diagram for reference. The most uncomfortable part of learning a complex system isn't not understanding a specific point, but not knowing where that point stands in the whole. So you can bookmark the diagram below and use it as a "navigation page" when reading the later articles.

catbuddy's harness consists of six major organs, wrapped in an outer layer of productization shell. Let's bring a real message "to life" and see which organs it alerts in sequence:

Just grasp one main axis: User message comes in → handed to the heart (AgentLoop) → in each round, the heart dispatches the eyes to assemble context, dispatches the hands and feet to execute tools, and queries the large model across the resilience layer → the result goes back out. The six organs each manage their own area, corresponding to the later articles:

Organ	Responsibility	Real Module	In Article No.
❤️ Heart	Drives the core loop of "ask→do→ask again"	loop.ts / runner.ts	02 · 03
🖐️ Hands & Feet	Allows the model to actually read/write files, run commands, connect external tools	tools/	04 · 05
👁️ Eyes	Determines "what the model can see, and how much" in each round	context/	06
💾 Memory	Remembers across sessions, distills conversations into long-term memory	memory.ts / dream.ts	07
🛡️ Resilience	Survives in the chaos of the real world	providers/ / hook.ts	08 · 09
🚀 Advanced	Sub-agent concurrency, introspection, letting AI draw diagrams	subagent.ts / self.ts	10 · 11

Behind this layering is a very hard design principle—each organ is only responsible to its upstream and downstream, mutually unaware of each other's internal details. For example, AgentRunner always gets the same LLMProvider interface; it fundamentally doesn't know whether it's calling Claude or DeepSeek, or whether it has already failed over to a backup model. So adding a new provider requires zero changes to the Runner. This is why this harness can simultaneously handle production loads and continuously add features—changes are locked inside a single organ, not pulling one hair and moving the whole body. This main thread will run through the entire series.

7. Three Common Misconceptions, Clarified in Passing

Misconception One: "Isn't a harness just writing a good prompt?" No. A prompt is the input fed to the model; it is a part of the harness (belonging to context engineering). But the harness also includes code logic outside the model, such as the loop, tool execution, and error recovery. No matter how good the prompt, it cannot make the model read a file by itself.

Misconception Two: "Don't LangChain / some framework provide ready-made ones?" Those frameworks help encapsulate a part, but the decisions—"how should your Agent manage context in what scenario, how to degrade, what tools to give"—cannot be replaced by a framework. The framework is the material; the harness is the house you build with the material. This series is about how to build.

Misconception Three: "If the model is strong enough, the harness isn't important." Quite the opposite. The stronger the model, the more you want to entrust it with more complex, longer, higher-risk tasks—and the more complex the task, the higher the requirements for loop stability, context management, and error recovery. As the model gets stronger, the harness doesn't become less important; it takes on heavier responsibility.

What did this article cover?

1. A bare LLM is a stateless text completion function with three inherent disabilities: no memory, can't take action, cannot self-advance. So it can chat, but it can't get work done independently.
2. The Harness is the engineering system that patches these three disabilities—using "stored history" to patch memory, "tools" to patch hands and feet, and a "loop" to patch self-advancement. Its core is the Agent Loop. The model is the engine; the harness is the chassis.
3. catbuddy's harness consists of six major organs (Heart/Hands & Feet/Eyes/Memory/Resilience/Advanced), and its soul is decoupling—each organ only recognizes interfaces, not implementations. Please bookmark this panoramic map; it is the navigation page for all subsequent articles.

Next Article: Enough concepts, time to get hands-on. In the next article, we'll set aside all of catbuddy's complexity and hand-write a minimal harness that can invoke tools from scratch in about 50 lines of code, letting you see with your own eyes how that "heart" starts beating on its own—and then you'll know exactly what catbuddy's thousand-line loop added on top of those 50 lines.