LLM · Agent · AI Programming

The Agent Harness: Why Prompt Engineering Alone Can't Build Reliable AI Agents

By Worlds · Jul 4, 2026

Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Most agent projects stall because teams treat the model as the whole system. A Harness is what makes the difference between a demo that works once and an agent that runs unattended across dozens of steps without burning tokens, corrupting state, or executing dangerous commands.

Summary

Single tool calls and clever prompts can't sustain complex agent tasks like cross-file refactoring or long-running research. The missing piece is a Harness—a state-machine-driven runtime that sits between the model and the execution environment. It decomposes agent behavior into standard states (think, call tool, wait, finish, abort) and drives a closed-loop execution cycle.

The Harness consists of five core modules: dynamic prompt assembly with layered caching, an execution loop engine with hard step/timeout limits, a tool scheduler that parses and validates calls before sandboxed execution, a context state manager that compresses and prioritizes information to prevent window overflow, and a security guardrail layer enforcing pre-validation, isolation, and audit logging. Claude Code and OpenAI Codex both implement these patterns, though Claude Code has recently stripped 80% of its system prompt to rely more on model-native reasoning.

Common failures—infinite loops, ignored errors, context amnesia—trace back to missing Harness components, not weak prompts. The engineering advice is to start with a minimal closed loop (prompt + one tool + simple cycle), then layer in state management, sandboxing, and caching based on real failure cases, while instrumenting every step for token cost and success-rate data.

Takeaways

— Single function-calling is not an agent; multi-step tasks require a runtime that manages iteration, state, and error recovery.

— A Harness is a state machine plus an execution loop—model outputs are parsed as state transitions, not treated as final answers.

— Dynamic prompt assembly uses three tiers with different cache lifetimes: static identity rules, semi-dynamic tool lists, and fully dynamic environment state.

— Execution loops need hard limits on steps and timeouts, plus tiered error handling that auto-retries transient failures and surfaces recoverable ones to the model.

— Tool scheduling must validate parameters, enforce permission tiers (auto, read-only, confirm), and format results before injecting them back into context.

— Context state management prevents window overflow by compressing old history, summarizing large files, and supporting checkpoint-based task resumption.

— Security requires pre-validation, sandbox isolation, and full audit logging—Claude Code restricts file ops to the working directory and forces confirmation on destructive commands.

— Claude Code's latest system prompt is 80% shorter, relying on model capability over exhaustive rules; prompt bloat introduces noise.

— OpenAI Codex shares one Harness across CLI, Web, VS Code, and desktop via JSON-RPC, with a compact API that compresses context into encrypted hidden state.

— Start with a minimal loop (prompt + one tool + simple cycle), then add modules based on real failure data, instrumenting token cost and success rate from day one.

Conclusions

Prompt engineering is less than 20% of the Harness engineering surface, yet it receives disproportionate attention because it's the most visible part.

Claude Code's 80% prompt reduction contradicts the common instinct to add more rules; stronger models need clearer boundaries, not denser instructions.

Tool-call parsing failures are often blamed on the model when the real culprit is a Harness that doesn't enforce a strict output schema with XML or JSON tags.

Codex's multi-end shared Harness via JSON-RPC is an architectural bet that agent logic should be a service, not a per-client implementation—this changes how teams should think about shipping agent features.

The compact API that replaces raw context with encrypted hidden state is an under-discussed privacy and cost lever: it keeps semantic continuity without shipping full conversation text to the model provider on every turn.

Concepts & terms

Agent Harness

A runtime control framework that wraps a large model, managing the full execution lifecycle—loop scheduling, tool dispatch, state persistence, and security—to turn open-ended model outputs into predictable, multi-step operations.

ReAct paradigm

A reasoning-and-acting loop pattern (Think → Act → Observe) that drives agent state transitions; the Harness implements this as a state machine, not the model itself.

Layered prompt assembly

A prompt construction strategy that separates static identity rules, semi-dynamic tool descriptions, and fully dynamic environment state into distinct segments with different cache lifetimes to reduce token cost.

Context compaction

An API endpoint (e.g., Codex's /responses/compact) that replaces overflowing conversation text with an encrypted hidden-state summary, preserving semantic continuity without shipping full history to the model.

Tool permission tiering

A security pattern that classifies tool operations by risk level—auto-execute, read-only, or require user confirmation—enforced by the Harness before any external action runs.

Source: juejin.cn ↗ Google Translate ↗ Backup ↗