跪拜 Guibai
← All articles
Artificial Intelligence · Open Source

Harness² Lets Agents Rewrite Their Own Prompts and Tools While Model Weights Stay Frozen

By 冬奇Lab ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Agent harness tuning has been a manual, trial-and-error craft; Harness² formalizes it into a recursive optimization that runs without ground-truth labels, but its default unrestricted network and disabled Codex sandbox mean any evaluation environment must be locked down first to get trustworthy results.

Summary

Harness², an open-source framework from Google Research, treats the agent's "shell"—system prompt, guardrails, memory, skills, tool guidance, scripts, subagents, and plugins—as eight editable components. With model weights fixed, a task-level recursion called Propose–Probe–Compose runs the agent on a task, proposes modifications, probes them on the same task, and synthesizes an improved harness by comparing behavior across executions, no ground truth required. A second, harness-level recursion maintains a persistent improvement harness that carries adopted harnesses and evidence-backed lessons forward to guide future tasks.

On three specialized benchmarks (JobBench, LAB, WorkBuddy-Bench), a single Harness² step lifts GLM-5 and Gemini 3.5 Flash above their base harnesses, with maximum gains of +11.5 points, +4.5 points, and +11.8 percentage points respectively. The README stresses that these are maximum improvements from the authors' own evaluations, not reproduced externally. Running real benchmarks requires Linux, Python 3.12, uv, a Vertex AI project, and costs API money per run. The demo mode runs without any model credentials, and the framework defaults to unrestricted network access with the Codex sandbox off, so the README explicitly orders network restrictions before evaluation to avoid agents reading public benchmark rubrics.

Takeaways
— Harness² is an Apache-2.0 licensed framework from Google Research with 23,451 stars and 2,354 forks as of the project's creation on 2026-10-06.
— It defines eight editable harness components: system prompt, guardrails, memory, skills, tool guidance, scripts, subagents, and plugins.
— Task-level recursion uses Propose–Probe–Compose: executing a task, proposing invariant and domain modifications, probing each candidate on the same task, and composing an improvement from behavioral comparisons without ground truth.
— Harness-level recursion maintains a persistent improvement harness that records adopted harnesses and evidence-backed lessons for future tasks.
— A single optimization step lifts GLM-5 and Gemini 3.5 Flash above base harnesses by up to +11.5 points on WorkBuddy-Bench, +4.5 points on JobBench, and +11.8 percentage points on LAB partial pass rate.
— Running real benchmarks requires Linux, Python 3.12, uv, and a Vertex AI project; model-based runs incur API costs, so the README advises starting small.
— The executor's network access is unrestricted by default and the Codex sandbox is off; the README instructs configuring network restrictions before evaluation to prevent agents accessing benchmark rubrics.
— The project has 163 open issues and the README disclaims that it is not an officially supported Google product.
— A credential-free local demo runs both recursion modes without calling any model, using the command "harness2-demo --output runs/demo --k 2".
Conclusions

The absence of ground-truth feedback is what makes Harness² suitable for open-world tasks, but it also means the quality of improvements hinges entirely on the quality of behavioral comparisons, which the README doesn't quantify beyond single-step gains.

Persisting improved harnesses and lessons across tasks could compound gains over time, yet the README shows no long-term data for that harness-level recursion, leaving the main claimed advantage partially unproven in the public material.

The safety warning is unusually explicit for a research repo: an agent that rewrites its own prompts and guardrails, combined with a disabled sandbox and unrestricted network, turns evaluation into a security exercise rather than just a performance test.

Concepts & terms
Harness
The system around a model that enables it to act as an agent: prompts, tools, memory, skills, orchestration logic, and other peripheral components that aren't part of the model's weights.
Harnessing
The process of optimizing that system — improving the harness configuration to make the agent perform better on tasks.
Propose–Probe–Compose
A task-level recursion pattern where the system proposes harness modifications, probes each by executing the agent on the same task, and composes a final harness by comparing behavioral differences across those executions. No ground-truth labels are needed.
Improvement Harness
A persistent, harness-level component that records adopted harnesses and evidence-backed lessons from completed tasks, then uses that accumulated experience to guide improvements for future tasks.
Source: juejin.cn ↗ Google Translate ↗ Backup ↗