Harness² Lets Agents Rewrite Their Own Prompts and Tools While Model Weights Stay Frozen
Open Source Project #258: Harness² —— Google Research's "Recursive Agent Harness Optimization", Model Weights Untouched, Only the Agent's Shell Changes
"A harness is the system around a model that enables it to act as an agent. Harnessing is the process of optimizing that system."
This is the #258 entry in the "One Open Source Project Per Day" series. Today's project is Harness² (pronounced Harness Squared) — Google Research's open-source recursive agent harness optimization framework, with 23,451 stars, 2,354 forks, Apache-2.0 license, created on October 6, 2026.
Over the past year it's become clear: the same model, wrapped in different "shells", can perform very differently. That shell is the Harness — prompts, tools, memory, skills, orchestration logic, and so on. But the shell is largely hand-tuned: try a version, run a few tasks, then change it again. Harness² asks: can an agent, when encountering a new task its harness never saw during design, improve its own shell without needing ground-truth feedback? Its approach does two levels of recursion while keeping model weights completely unchanged.
What you will learn
- The difference between Harness and Harnessing, and the eight editable components Harness² exposes
- The two levels of recursion: task-level "Propose–Probe–Compose" and harness-level "improving the Harness"
- The difference between parallel and sequential recursion, and how to use the
--kbudget - Quick start: a local demo that requires no model credentials, and the full command to run one LAB benchmark
- Two things to know before using: API costs, and network access risks during evaluation
Prerequisites
- Basic understanding of AI agents and tool calling
- Knowing coding agent executors like Claude Code, Codex, OpenCode makes the integration easier to grasp
- Running benchmarks locally requires Linux, Python 3.12, and uv; real benchmarks require a Vertex AI project
Project Background
Overview
Harness² is a plug-and-play framework targeting open world scenarios: agents encounter tasks the harness didn't anticipate during design. With fixed model weights, it connects two levels of recursion:
- Task-level recursion: For the current task, it improves the execution harness through a comparative Propose–Probe–Compose step — proposes modifications, probes them on the same task, then composes an improved harness based on behavioral comparisons between different executions. This requires no ground-truth feedback.
- Harnessing-level recursion: Maintains a persistent improvement harness that writes adopted harnesses and evidence-backed lessons from each task into it, to guide later improvements.
Author / Team
- Developer: Google Research (
google-researchorganization). Paper authors include Ruiyao Xu, Yanfei Chen, Zhongying CuiZhu, Bhavana Dalvi Mishra, Yifei Ming, Han Yu, Rujun Han, Chen-Yu Lee, Tomas Pfister - Paper: arXiv:2610.14224 "Harness²: Recursive Agent Harnessing for an Open World"
- Primary language: Python
- License: Apache-2.0;
third_party/, benchmarks, and datasets retain their upstream licenses - Created: 2026-10-06, last commit 2026-11-02
- Disclaimer: The README explicitly states "This is not an officially supported Google product" and it is not covered by Google's Open Source Vulnerability Rewards Program
Project Data
- ⭐ GitHub Stars: 23,451+
- 🍴 Forks: 2,354+
- 📄 License: Apache-2.0
- 📅 Created: 2026-10-06
- 📬 Open Issues: 163
- 👥 Contributor commits are spread out: rudycelekli 8 times, most others 1–2 times, suggesting major development happened outside the repo and was released as a whole
- 🌐 Project site: harness2.github.io
Quick Start
Requires Linux, Python 3.12, and uv.
1. Local demo (no model credentials needed)
uv venv --python 3.12 .venv
source .venv/bin/activate
uv pip install -e '.[render]'
harness2-demo --output runs/demo --k 2
The demo runs both recursion modes with a deterministic editor and real Python execution, without calling any model. To understand the flow first, this is the cheapest place to start.
2. Run a real benchmark
Needs Node.js 20+, Google Cloud CLI, and each benchmark's own dependencies (LAB needs Pandoc and Poppler, WorkBuddy-Bench needs Docker). Then configure models:
cp .env.example .env
# Fill in Vertex AI project and model ID in .env
gcloud auth application-default login
README's example runs a single LAB (legal work) task, then judges and produces a report:
harness2-setup lab
harness2-check --bench lab
harness2 --bench lab --k 1 --mode parallel \
--only-domains antitrust-competition --max-tasks 1
harness2-judge --bench lab --k 1 --mode parallel
harness2-report --bench lab --k 1 --mode parallel
Key points: --k is the task-level recursion budget; judging and reporting must use the same benchmark, executor, subset, --k, --mode, and random seed as the optimization run; adding --dry-run only previews task selection without calling models. The README explicitly warns: model-based runs incur API costs, so start with a very small task selection.
Core Analysis
Eight editable components: What can the Harness change?
Harness² splits the model's periphery into eight editable components, grouped by runtime role:
| Role | Component |
|---|---|
| Construct context | system prompt, guardrails, memory, skills |
| Define tool interface | tool guidance, scripts |
| Orchestrate delegation | subagents |
| Control middleware | plugins |
This split's significance: it turns "changing an agent" from a fuzzy manual craft into editing operations on a clear set of objects. The optimizer doesn't change the model; it changes these eight things, so results can be saved, reviewed, and reused.
Task-level recursion: Propose–Probe–Compose
For a specific task, a Propose–Probe–Compose step works like this:
- Execute: The agent executor runs the task with the initial harness.
- Propose: The harness editor proposes two types of changes — invariant changes and domain changes.
- Probe: Each candidate change is actually run on the same task to see behavioral changes.
- Compose: The harness composer synthesizes the final harness based on behavioral comparisons between these executions.
The key is steps 3 and 4 use "comparisons between different executions on the same task" rather than right/wrong judgments, so no ground truth is needed. The cost is direct: each probe is a real agent execution, so costs scale with the budget.
Task-level recursion budget has two ways to expand:
| Option | Behavior |
|---|---|
--k 1 |
One Propose–Probe–Compose step |
--k K --mode parallel |
Probe K generic and K domain-specific candidates, then produce K composed results (wider exploration) |
--k K --mode sequential |
Carry the improved harness and accumulated evidence into K consecutive steps (deeper improvement) |
--substrate cx |
Use Codex instead of the default OpenCode as the executor |
Harnessing-level recursion: Accumulating experience
After each task, the adopted harness and "evidence-backed lessons" are written into the persistent improvement harness, which guides improvements for subsequent tasks. The corresponding role in the flowchart is the "experience reflector": it reflects on the step and updates the improvement harness for the next task.
This layer is what distinguishes the framework from "hand-tuning each task from scratch." One thing to note: the README shows results mostly from single steps; I didn't see separate data on the long-term accumulation effect of this layer. For fuller ablations, see the paper.
Results from the paper, and how to read them
The README's conclusion: on three specialized benchmarks, across four model–harness configurations, a single Harness² step lets GLM-5 and Gemini 3.5 Flash surpass their base harnesses, with maximum improvements:
- WorkBuddy-Bench: up to +11.5 points
- JobBench: up to +4.5 points
- LAB partial pass rate: up to +11.8 percentage points
It also states increasing the task-level recursion budget brings further gains. Supported benchmarks are JobBench (professional work), LAB (legal work), and WorkBuddy-Bench (office, web, and code). Executors support OpenCode and Codex; recursion modes support parallel and sequential.
Two things to keep in mind when reading these numbers. First, these are "maximum improvements", not every configuration or benchmark achieves that magnitude. Second, these numbers are from the paper and the authors' own evaluations; I have not reproduced them.
A safety note you must read: Network access during evaluation
The README's "Web access" section is blunt: The executor's network access is unrestricted by default, and the Codex runtime sandbox is off. Public benchmark rubrics may be findable online. So before evaluation, you must configure network restrictions per docs/EXPERIMENTS.md to prevent agents from accessing benchmark answers and rubrics; otherwise results are untrustworthy.
This isn't just an academic honesty issue: an agent that can modify its own prompts, guardrails, and scripts, combined with no sandbox and unrestricted network, deserves careful thought about what it can touch before running on your own machine.
Project Address & Resources
- 🌟 GitHub: google-research/harness2
- 📄 Paper: arXiv:2610.14224
- 🌐 Project site: harness2.github.io
- 📖 Benchmark guide: docs/BENCHMARKS.md
- 🧪 Experiment guide: docs/EXPERIMENTS.md
- ⚙️ Setup guide: docs/SETUP.md
Summary
Harness² turns "tuning an agent" from a craft into a recursively executable optimization process: the model stays put, the harness is editable, saveable, and reusable.
Three points worth noting:
The optimization target is the harness, not the model. With weights fixed, eight well-defined editable components form the search space, making results reviewable and transferable, and applicable to closed-source models you can't fine-tune.
No ground truth; improvements come from "behavioral comparison". Propose–Probe–Compose synthesizes new harnesses by comparing differences across executions on the same task. This is what lets it handle open-world tasks, and also the source of its cost: every probe is a real execution.
Capability and risk are two sides of the same coin. It defaults to unrestricted network, Codex sandbox off, and real API costs; the README itself advises configuring network restrictions before evaluation. Before plugging it into real environments, first use the credential-free harness2-demo to understand the flow, then start from the smallest task selection. Also, this is research code; the README states it's not an officially supported product, and the repo has 163 open issues.
If you're researching agent self-improvement, want to compare harness performances, or are evaluating whether "changing the shell" or "swapping the model" is more cost-effective, this project is a reference-worthy starting point.
Explore PrimeSkills — curated AI agent and skill tools, each verified through real workflows. No hype, only genuinely useful tools.
Visit my personal site for more insights and interesting products.