Building an Agent Harness from Scratch: Loops, Constraints, and Context Engineering

This article represents only the author's personal views. If there are errors or differences in understanding, feedback and discussion are welcome.

The learning process referenced the source code of Pi Agent. Following its layered approach and some implementation ideas, the author independently implemented an MVP version of a Harness.

This article mainly shares some learning insights and personal thoughts from the entire process.

Preface

"Agent" has evolved from initial text-based dialogue, to multimodal (voice/image/video) agents, and now to Agents (capable of assisting with office work, development, generating complete projects, etc.). The definition of Agent differs at each stage.

The most recent stage is "Model + Harness = Agent".

Previously, I frequently used various agents to assist with writing documents and modifying project code.

Recently, I've been preparing to build some AI applications with Agent capabilities, so I'm first learning about Harness-related knowledge — know yourself and know your enemy 😋.

Let's start with a diagram

Harness is essentially the model's runtime. The differences between various Agents lie in this part; the model itself can be swapped at any time.

So an 'Agent' without a Harness is mostly just a chatbot.

A quick summary of the main modules

Module	Function
Model	Reads Context, produces text or structured tool_calls
Context	The actual content fed into the model for this turn, not just 'the user's single input sentence'
Tools	Performs actual external calls, truly modifies files, runs commands
Harness	Responsible for looping, validation, execution, context assembly, write-back, permissions, resource control, and other behaviors
Agent	The model runs on the Harness, advancing step-by-step toward a goal until completion or a termination condition is met

Tools

Tools are the bridge between the model and the external world: the model only issues structured calls; the actual file modification, command execution, and API calls are done by the specific Tool implementation; scheduling and validation are handled by the Harness.

Function Calling First

Function Calling (now often also called Tool Calling) solves the problem of:

Preventing the model from describing the tool to call in natural language, and instead having it output a parseable call structure.

Without this convention, the model might write:

I need to call get_weather(Shanghai) to get the weather information

It's difficult for the application side to parse this reliably. With Function Calling:

1. First, provide the tool's schema to the model (what can be called, what the parameters look like)
2. When needed, the model returns tool_calls (who to call, what the parameters are, the ID of this call)
3. The Harness parses the call, executes the tool, and writes the result back as a tool message
4. The Harness then requests the model again to continue reasoning based on the tool execution result

Here's an official diagram: function-calling#how-it-works

Below is a diagram I drew of how Model / Harness / Tools flow together (corresponding to steps 1→4 above):

Call Example

Here's a minimal demo (OpenAI Chat Completions, non-streaming) to look at the call structure:

// 1) Prepare tool declarations (schema) — only describes capabilities, no implementation
const tools = [
  {
    type: 'function',
    function: {
      name: 'get_weather',
      description: 'Query the current weather for a city',
      parameters: {
        type: 'object',
        properties: {
          city: { type: 'string', description: 'City name, e.g., Shanghai' },
        },
        required: ['city'],
        additionalProperties: false,
      },
    },
  },
]

const messages = [
  { role: 'user', content: 'What is the weather like in Shanghai today?' },
]

// 2) One request: messages + tools
const res = await fetch(`${OPENAI_API_BASE_URL}/v1/chat/completions`, {
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${OPENAI_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'gpt-5.6', // model id
    messages,
    tools,
    tool_choice: 'auto', // model decides whether to call a tool; can also be none / required / specify a tool
  }),
})

const data = await res.json()
const message = data.choices[0].message
// ↓ Output explanation see example below

A successful response (when a tool call is needed) looks roughly like this:

// data illustration (omitting usage, etc.)
{
  id: "chatcmpl-xxx",
  choices: [
    {
      index: 0,
      finish_reason: "tool_calls", // This turn ended due to a tool call; final text often shows "stop"
      message: {
        role: "assistant",
        content: null, // Could also be a description; often null when tool_calls exist
        tool_calls: [
          {
            id: "call_abc123",
            type: "function",
            function: {
              name: "get_weather",
              arguments: '{"city":"Shanghai"}', // Note: it's a string, not an object
            },
          },
        ],
      },
    },
  ],
}

In the returned result, focus on these:

• message.tool_calls: Determines if a tool needs to be executed
• tool_calls[].id: Must correspond one-to-one when writing back results
• tool_calls[].function.name / arguments: Tool name and parameters
• finish_reason: "tool_calls" means 'tool execution needed'; "stop" means this turn is over

Parse and Execute Example

The core logic for the Harness to handle tool calls is: Validate → Execute → Write Back → Request Again.

// 1) First, push the current assistant message (including tool_calls) into messages for the next turn
// Keep the original OpenAI structure
messages.push({
  role: 'assistant',
  content: message.content,
  tool_calls: message.tool_calls,
})

// 2) Parse parameters → Validate → Execute → Write Back
for (const tc of message.tool_calls) {
  // Format the parameters
  const call = {
    id: tc.id,
    name: tc.function.name,
    arguments: JSON.parse(tc.function.arguments || '{}'), // string → object
  }

  // Validate against schema; write back an error if invalid
  if (isInvalid(call)) {
    messages.push({
      role: 'tool',
      tool_call_id: call.id,
      content: 'Invalid parameters',
    })
    continue
  }

  // Execute the tool
  const result = await executeTool(call)

  // Write the result back into the context
  messages.push({
    role: 'tool',
    tool_call_id: call.id,
    content: result.content, // string
  })
}

// 3) Request the model again with the updated messages
const res = await fetch(`${OPENAI_API_BASE_URL}/v1/chat/completions`, {
  // ... other parameters
  body: JSON.stringify({ messages, tools, tool_choice: 'auto' }),
})

const data = await res.json()
// The model replies again, possibly containing tool_calls or continuing with a natural language response
const message = data.choices[0].message

Below is an example of the messages for the second model request, including the tool call result:

[
  { "role": "user", "content": "What is the weather like in Shanghai today?" },
  {
    "role": "assistant",
    "content": null,
    "tool_calls": [
      {
        "id": "call_abc123",
        "type": "function",
        "function": {
          "name": "get_weather",
          "arguments": "{\"city\":\"Shanghai\"}"
        }
      }
    ]
  },
  {
    "role": "tool",
    "tool_call_id": "call_abc123",
    "content": "{\"temp\":28,\"unit\":\"C\"}"
  }
]

Summary

1. Function Calling enables the model to output parseable tool_calls parameters
2. Tool: Composed of a declaration (schema) and an implementation (handler); the declaration is given to the model, the implementation stays in the Harness
3. Harness is responsible for validation → execution → writing back by tool_call_id → requesting again

Agent Loop

An Agent that actually gets things done needs to automatically loop the above process: assemble context → request model → if there's a call, execute and write back → request again, until a final answer is given or a constraint forces it to stop.

Of course, what I have here is just the simplest implementation (using 'whether there are tool_calls' to decide whether to auto-advance); actual loop continuation conditions might have other strategies.

Simplified Loop Illustration

async function runAgent(state) {
  while (true) {
    // 1) Assemble the Context that actually goes into the model for this turn
    const messages = assembleContext(state)

    // 2) Request the model
    const assistant = await adapter.stream({ messages, tools: state.tools })

    // 3) Check if there are tool_calls
    if (!assistant.tool_calls?.length) {
      return assistant.content
    }

    // 4) There are calls: Validate → Execute → Write Back
    state.messages.push({
      role: 'assistant',
      content: assistant.content,
      tool_calls: assistant.tool_calls,
    })

    for (const call of assistant.tool_calls) {
      const result = await executeTool(call)
      state.messages.push({
        role: 'tool',
        tool_call_id: call.id,
        content: result.content,
      })
    }
    // 5) Enter the next while loop with the tool results
  }

  return assistant.content
}

The difference from pure dialogue lies in step 4: the model outputs a call instruction, the Harness schedules the Tools, and uses the execution results to continue advancing.

Other Conditions

Taking Pi: agent-loop as an example, the default is still 'whether there were tool calls in this turn' to drive automatic continuation, while also checking the following conditions:

1) Tool results may not need to be returned

• The tool execution result includes a terminate: true field to decide whether to return it
• Sometimes the final answer is already in the tool result (e.g., a structured output tool), saving a round of LLM to 'rephrase' the result
• If all tool calls in this turn have terminate: true, there's no need to request the model again

// Illustration: A structured output tool ends the automatic continuation for this turn upon completion
async execute(_id, params) {
  return {
    content: [{ type: "text", text: "Structured result saved" }],
    details: params,
    terminate: true,
  };
}

2) shouldStopAfterTurn (Optional stop after a turn)

Built-in hook: After a turn ends, the caller decides whether to continue; context can also be selectively compressed here.

3) steering / follow-up

Both are 'injecting user messages into the Loop', but at different times:

Term	Plain English	Typical Timing
steering	While the Agent is still running, the user can temporarily insert a sentence to change the execution direction	Injected after the current turn's tools finish, before the next model request
follow-up	The Agent is about to finish, but there's still a sentence in the queue, like 'Also, summarize it'	Taken from the queue before the inner loop exits; if present, continue running

The related loop code is about 100 lines, relatively easy to understand.

Summary

1. The minimal Loop model = Assemble context → Call model → (If tool_calls exist) Execute tools → Request again for the next round
2. Traces can be recorded within the Loop, exposing relevant information through events, and listened to outside the Loop for further analysis.

Runtime Constraints

Once the Loop can run, the next step is: how to make it stop.

Prompts can only provide soft constraints: irreversible operations, infinite call loops, repeated executions, etc., require hard constraints in the Harness code.

Common Constraints

const controller = new AbortController()

await runAgent({
  maxSteps: 8, // Maximum number of execution turns
  timeoutMs: 60_000, // Timeout for the entire Run
  stopOnToolError: false, // false: write back the error, let the model correct it
  signal: controller.signal,
  onConfirm: async (call) => {
    // Secondary confirmation before executing high-risk tools; rejection writes an error tool result without calling the real handler
    if (call.name === 'rm' || call.name === 'bash') {
      return window.confirm(`Allow execution of ${call.name}?`)
    }
    return true
  },
})

// controller.abort(); Active cancellation

Validation

Add a pre-execution validation layer before executing tool calls: validate the schema, check if the tool exists, check if parameters are valid, etc.

Abnormal situations can write back an error Tool message, allowing the model to autonomously retry, and the user can also perceive it.

Summary

Loop termination and tool call safety boundaries rely on hard constraints in the Harness code.

Execution sandboxes and the like will be studied carefully later, to be covered in a separate article.

Context Engineering

Context assembly is also the application of Context Engineering within the Harness.

What is pushed to the model each time is not just 'the single sentence the user just input', but the complete context assembled by the Harness, commonly including:

1. System prompt: Role, global behavior; project Rules are often merged into this layer
2. Conversation history: Full text, or first summarize the historical conversation content, then keep the most recent rounds
3. tool schemas: Tool declarations (local + MCP unified into the same schema set)
4. Tool call results: role: "tool" messages written back from the previous/current Loop turn
5. Current user input: The current intent
6. On-demand materials: Memory (cross-session memory), RAG (retrieved content snippets), SKILL (SOP instructions), etc., injected only when relevant

Illustrative Code

const context = assembleContext({
  system: [identity, ...rules], // ① System prompt + Rules
  tools: toolDefs, // ③ schemas (including MCP normalized ones)
  history: messages, // ②④ History already contains tool results; or summarize first
  user: currentUserMessage, // ⑤ Current turn input
  // skillCatalog / skillBody     // ⑥ SKILL: SKILL catalog and related SKILL description, full text injected on demand
  // memoryHits / ragChunks       // ⑥ Memory / RAG: External hits → inject
})

Summary

1. Persistent content: System prompt / history summary / tool schemas / tool results / current turn user;
2. On-demand content: Memory, RAG, SKILL, etc., injected only when relevant

The detailed implementation of this part requires further study of several projects, to be covered in a separate article.

SKILL

Can be seen as a process specification document (SOP) for solving a certain type of problem or completing a certain task.

SKILL is not a Tool; it can be viewed as a large block of prompt text. Actual execution still completes tasks by calling Tools through internal instruction descriptions.

SKILL Structure

Commonly a Markdown file (possibly with frontmatter), roughly three parts:

Part	Content
Metadata	name, description (when to use); optional permissions (allowed tools) / dependency descriptions
Body	Steps, constraints, output format, decision rules (the real SOP)
Other	(Optional) references/, templates, scripts, workflows, etc., this part is generally progressively disclosed

---
name: weather-brief
description: Use when the user wants a weather briefing
---

1. First call get_weather …
2. Then output in a fixed structure …

How it Enters Context

Several methods learned:

Method	Who decides to load the full text	How the full text enters
Manual / Explicit	User specifies (e.g., /skill:name)	Directly written into Context
Harness Match	Simple keyword matching of SKILL description against user input	The matched body text is written into Context
Model Decides	The model chooses which one to load	First, give the model the SKILL catalog (name + description); after the model decides, it calls the tool load_skill, and the SKILL body text is written back as a tool result

The step-by-step flow for the three methods (using the same example weather-brief):

Summary

SKILL can be discovered and injected into the context by the Harness, or some basic information can be disclosed to the model, letting the model decide which one to load.

MCP

MCP (Model Context Protocol) is an open protocol for standardizing how AI applications connect to external systems (data, tools, etc.).

This article first only looks at the Tools part: unified discovery and invocation.

The execution path after receiving tool_calls:

Harness → Select Client → Server → The returned result is written back to Context.

Role	Responsibility
MCP Host	Creates/manages multiple Clients; here, the Harness is responsible for management
MCP Client	A session connection with a single Server: provides common methods like listTools, callTool
MCP Server	Exposes Tools and executes them (local process or remote service; also has Resources / Prompts, not expanded here)

Transport: Locally, stdio is mostly used; remotely, Streamable HTTP is mostly used; the data transfer protocol uses JSON-RPC.

MCP Server (Tools)

// mcp-server.mjs —— This is the process the Host spawns
import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js'
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js'
import { z } from 'zod'

const server = new McpServer({ name: 'demo', version: '1.0.0' })

server.registerTool(
  'add',
  {
    description: 'Calculate the sum of two numbers',
    inputSchema: {
      a: z.number().describe('Addend a'),
      b: z.number().describe('Addend b'),
    },
  },
  async ({ a, b }) => ({
    content: [{ type: 'text', text: String(a + b) }],
  }),
)

const transport = new StdioServerTransport()
await server.connect(transport) // stdout is dedicated to JSON-RPC; use console.error for logs

Integrating into the Harness

The registry mentioned below can be seen as the module within the Harness responsible for managing tool registration and discovery

Includes management of local Tools + MCP Tools

import { Client } from '@modelcontextprotocol/sdk/client/index.js'
import { StdioClientTransport } from '@modelcontextprotocol/sdk/client/stdio.js'

// Connect to MCP Server
const transport = new StdioClientTransport({
  command: 'node',
  args: ['./mcp-server.mjs'],
})
const client = new Client({ name: 'my-harness', version: '0.1.0' })
await client.connect(transport) // spawn + initialize

const { tools } = await client.listTools()
for (const t of tools) {
  // Write into the Harness tool table
  registry.register({
    name: t.name,
    description: t.description ?? '',
    parameters: t.inputSchema ?? { type: 'object', properties: {} },
    source: { kind: 'mcp', client },
  })
}

async function executeTool(call: ToolCall): Promise<ToolResult> {
  const meta = registry.get(call.name)
  if (meta?.source.kind === 'mcp') {
    const res = await meta.source.client.callTool({
      name: call.name,
      arguments: call.arguments,
    })
    return {
      toolCallId: call.id,
      content: JSON.stringify(res.content),
      isError: Boolean(res.isError),
    }
  }
  return localHandlers[call.name](call)
}

The Client connection is long-lived:

• At startup: new Client → connect → listTools registers tools once;
• Subsequently, callTool is called repeatedly across multiple Agent Loop turns, reusing the previous connection;
• Generally, close() is only called when the Harness exits or the user actively stops that Server.

MCP also includes Resources and Prompt capabilities; this part needs further study on the linkage mechanism with the Harness

Finally

At this point, building an MVP runnable Agent from 0 to 1 should be solid.

The next part (Part 2) will likely cover the following content:

1. Memory / RAG: How to retrieve, when to inject, how to trim; plan to look at the implementation details of major Agent frameworks and compare effects with some industry open-source libraries
2. Execution Sandbox and Permission Control: How to isolate some tool executions from the user's project environment; tool whitelists, execution permission control, etc.
3. Other MCP Content: Resources / Prompts

Related Links

• Experimental Project: notes/my-harness
• PI Agent: pi/agent