跪拜 Guibai
← Back to the summary

Harness² Lets Agents Rewrite Their Own Prompts and Tools While Model Weights Stay Frozen

Open Source Project #258: Harness² —— Google Research's "Recursive Agent Harness Optimization", Model Weights Untouched, Only the Agent's Shell Changes

"A harness is the system around a model that enables it to act as an agent. Harnessing is the process of optimizing that system."

This is the #258 entry in the "One Open Source Project Per Day" series. Today's project is Harness² (pronounced Harness Squared) — Google Research's open-source recursive agent harness optimization framework, with 23,451 stars, 2,354 forks, Apache-2.0 license, created on October 6, 2026.

Over the past year it's become clear: the same model, wrapped in different "shells", can perform very differently. That shell is the Harness — prompts, tools, memory, skills, orchestration logic, and so on. But the shell is largely hand-tuned: try a version, run a few tasks, then change it again. Harness² asks: can an agent, when encountering a new task its harness never saw during design, improve its own shell without needing ground-truth feedback? Its approach does two levels of recursion while keeping model weights completely unchanged.

What you will learn

Prerequisites


Project Background

Overview

Harness² is a plug-and-play framework targeting open world scenarios: agents encounter tasks the harness didn't anticipate during design. With fixed model weights, it connects two levels of recursion:

Author / Team

Project Data


Quick Start

Requires Linux, Python 3.12, and uv.

1. Local demo (no model credentials needed)

uv venv --python 3.12 .venv
source .venv/bin/activate
uv pip install -e '.[render]'
harness2-demo --output runs/demo --k 2

The demo runs both recursion modes with a deterministic editor and real Python execution, without calling any model. To understand the flow first, this is the cheapest place to start.

2. Run a real benchmark

Needs Node.js 20+, Google Cloud CLI, and each benchmark's own dependencies (LAB needs Pandoc and Poppler, WorkBuddy-Bench needs Docker). Then configure models:

cp .env.example .env
# Fill in Vertex AI project and model ID in .env
gcloud auth application-default login

README's example runs a single LAB (legal work) task, then judges and produces a report:

harness2-setup lab
harness2-check --bench lab
harness2 --bench lab --k 1 --mode parallel \
  --only-domains antitrust-competition --max-tasks 1
harness2-judge --bench lab --k 1 --mode parallel
harness2-report --bench lab --k 1 --mode parallel

Key points: --k is the task-level recursion budget; judging and reporting must use the same benchmark, executor, subset, --k, --mode, and random seed as the optimization run; adding --dry-run only previews task selection without calling models. The README explicitly warns: model-based runs incur API costs, so start with a very small task selection.


Core Analysis

Eight editable components: What can the Harness change?

Harness² splits the model's periphery into eight editable components, grouped by runtime role:

Role Component
Construct context system prompt, guardrails, memory, skills
Define tool interface tool guidance, scripts
Orchestrate delegation subagents
Control middleware plugins

This split's significance: it turns "changing an agent" from a fuzzy manual craft into editing operations on a clear set of objects. The optimizer doesn't change the model; it changes these eight things, so results can be saved, reviewed, and reused.

Task-level recursion: Propose–Probe–Compose

For a specific task, a Propose–Probe–Compose step works like this:

  1. Execute: The agent executor runs the task with the initial harness.
  2. Propose: The harness editor proposes two types of changes — invariant changes and domain changes.
  3. Probe: Each candidate change is actually run on the same task to see behavioral changes.
  4. Compose: The harness composer synthesizes the final harness based on behavioral comparisons between these executions.

The key is steps 3 and 4 use "comparisons between different executions on the same task" rather than right/wrong judgments, so no ground truth is needed. The cost is direct: each probe is a real agent execution, so costs scale with the budget.

Task-level recursion budget has two ways to expand:

Option Behavior
--k 1 One Propose–Probe–Compose step
--k K --mode parallel Probe K generic and K domain-specific candidates, then produce K composed results (wider exploration)
--k K --mode sequential Carry the improved harness and accumulated evidence into K consecutive steps (deeper improvement)
--substrate cx Use Codex instead of the default OpenCode as the executor

Harnessing-level recursion: Accumulating experience

After each task, the adopted harness and "evidence-backed lessons" are written into the persistent improvement harness, which guides improvements for subsequent tasks. The corresponding role in the flowchart is the "experience reflector": it reflects on the step and updates the improvement harness for the next task.

This layer is what distinguishes the framework from "hand-tuning each task from scratch." One thing to note: the README shows results mostly from single steps; I didn't see separate data on the long-term accumulation effect of this layer. For fuller ablations, see the paper.

Results from the paper, and how to read them

The README's conclusion: on three specialized benchmarks, across four model–harness configurations, a single Harness² step lets GLM-5 and Gemini 3.5 Flash surpass their base harnesses, with maximum improvements:

It also states increasing the task-level recursion budget brings further gains. Supported benchmarks are JobBench (professional work), LAB (legal work), and WorkBuddy-Bench (office, web, and code). Executors support OpenCode and Codex; recursion modes support parallel and sequential.

Two things to keep in mind when reading these numbers. First, these are "maximum improvements", not every configuration or benchmark achieves that magnitude. Second, these numbers are from the paper and the authors' own evaluations; I have not reproduced them.

A safety note you must read: Network access during evaluation

The README's "Web access" section is blunt: The executor's network access is unrestricted by default, and the Codex runtime sandbox is off. Public benchmark rubrics may be findable online. So before evaluation, you must configure network restrictions per docs/EXPERIMENTS.md to prevent agents from accessing benchmark answers and rubrics; otherwise results are untrustworthy.

This isn't just an academic honesty issue: an agent that can modify its own prompts, guardrails, and scripts, combined with no sandbox and unrestricted network, deserves careful thought about what it can touch before running on your own machine.


Project Address & Resources


Summary

Harness² turns "tuning an agent" from a craft into a recursively executable optimization process: the model stays put, the harness is editable, saveable, and reusable.

Three points worth noting:

The optimization target is the harness, not the model. With weights fixed, eight well-defined editable components form the search space, making results reviewable and transferable, and applicable to closed-source models you can't fine-tune.

No ground truth; improvements come from "behavioral comparison". Propose–Probe–Compose synthesizes new harnesses by comparing differences across executions on the same task. This is what lets it handle open-world tasks, and also the source of its cost: every probe is a real execution.

Capability and risk are two sides of the same coin. It defaults to unrestricted network, Codex sandbox off, and real API costs; the README itself advises configuring network restrictions before evaluation. Before plugging it into real environments, first use the credential-free harness2-demo to understand the flow, then start from the smallest task selection. Also, this is research code; the README states it's not an officially supported product, and the repo has 163 open issues.

If you're researching agent self-improvement, want to compare harness performances, or are evaluating whether "changing the shell" or "swapping the model" is more cost-effective, this project is a reference-worthy starting point.


Explore PrimeSkills — curated AI agent and skill tools, each verified through real workflows. No hype, only genuinely useful tools.

Visit my personal site for more insights and interesting products.