Harness² Lets Agents Rewrite Their Own Prompts and Tools While Model Weights Stay Frozen
Agent harness tuning has been a manual, trial-and-error craft; Harness² formalizes it into a recursive optimization that runs without ground-truth labels, but its default unrestricted network and disabled Codex sandbox mean any evaluation environment must be locked down first to get trustworthy results.
Harness², an open-source framework from Google Research, treats the agent's "shell"—system prompt, guardrails, memory, skills, tool guidance, scripts, subagents, and plugins—as eight editable components. With model weights fixed, a task-level recursion called Propose–Probe–Compose runs the agent on a task, proposes modifications, probes them on the same task, and synthesizes an improved harness by comparing behavior across executions, no ground truth required. A second, harness-level recursion maintains a persistent improvement harness that carries adopted harnesses and evidence-backed lessons forward to guide future tasks.
On three specialized benchmarks (JobBench, LAB, WorkBuddy-Bench), a single Harness² step lifts GLM-5 and Gemini 3.5 Flash above their base harnesses, with maximum gains of +11.5 points, +4.5 points, and +11.8 percentage points respectively. The README stresses that these are maximum improvements from the authors' own evaluations, not reproduced externally. Running real benchmarks requires Linux, Python 3.12, uv, a Vertex AI project, and costs API money per run. The demo mode runs without any model credentials, and the framework defaults to unrestricted network access with the Codex sandbox off, so the README explicitly orders network restrictions before evaluation to avoid agents reading public benchmark rubrics.
The absence of ground-truth feedback is what makes Harness² suitable for open-world tasks, but it also means the quality of improvements hinges entirely on the quality of behavioral comparisons, which the README doesn't quantify beyond single-step gains.
Persisting improved harnesses and lessons across tasks could compound gains over time, yet the README shows no long-term data for that harness-level recursion, leaving the main claimed advantage partially unproven in the public material.
The safety warning is unusually explicit for a research repo: an agent that rewrites its own prompts and guardrails, combined with a disabled sandbox and unrestricted network, turns evaluation into a security exercise rather than just a performance test.