跪拜 Guibai
← All articles
Frontend · Backend · JavaScript

AI Agents Don't Need a Smarter Model—They Need a Runtime That Enforces 'Done'

By LVZ ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

The industry's verification harnesses all solve running an agent, not verifying one—none define 'done' as something you can check. GateLoop offers a concrete pattern for turning subjective agent claims into deterministic, script-checkable gates, which is the missing layer for unattended agents in CI pipelines or long-running coding tasks.

Summary

Agents keep claiming "verification passed" without running tsc, losing track of state by round 40, and burning tokens on the same error 20 times. The root cause, according to a new project called GateLoop, is that execution discipline lives in system prompts, which an LLM treats as suggestions it can selectively ignore. GateLoop is a runtime harness that replaces prompt-based appeals with enforced gates: each stage in a skill's manifest has verify hooks that re-run real tools like tsc, eslint, and prettier, advancing only on exit 0.

The engine, published as the npm package ai-rail (v0.0.0-beta.8, bin command nut), provides a state-machine flow with anchors, data contracts via emits/consumes, per-stage maxAttempts circuit breakers, and a flow-level maxJumps cycle detector. It also supports multi-skill orchestration through readsContext, writesMemory, and onEscalate rollback. A key design choice is that the model cannot write the state file; only the engine can. The project includes reproducible demos showing a bare SKILL.md-based agent missing a lint error while the GateLoop version catches all three types of issues through exit-code gates.

Takeaways
— GateLoop is a runtime harness (npm package ai-rail, v0.0.0-beta.8, bin command nut) that enforces agent execution via state machines, flow gates, and exit-code verification.
— Skills declare anchors with verify hooks that re-run real commands like tsc, eslint, and prettier; only exit 0 advances to the next stage.
— The engine, not the model, is the sole writer of the state file (anchor-state.json), preventing the model from faking progress.
— Per-stage maxAttempts acts as a circuit breaker; exceeding it escalates the anchor and stops token burn.
— Flow-level maxJumps detects onPass jumps that form cycles (A→B→A) and halts them.
— Data contracts via emits/consumes turn implicit handoffs into explicit checks: declared artifacts that aren't produced fail verification.
— In a reproducible demo, a bare SKILL.md skill declared 'fixed' but missed a lint error; the GateLoop version caught all three seeded issues through separate gates.
— The GateLoop boundary stops at release: scripts and models automate dev and test, but production release stays manual.
— Reproducible demos live in demo/gateloop-story/ and demos/gateloop/ in the repo history; source is at github.com/Lycorxy/gateloop.
Conclusions

The core reframe is that you can't force a model to execute, but you can make non-execution fail to advance—so the incentive structure changes from 'should obey' to 'can't get anywhere by lying'.

The five-way division of verification (exit code for determinism, schema for completeness, scripts+model for metrics, model+form-check review for subjective, and excluding pure aesthetics) is a practical answer to where AI judgment is and isn't trustworthy.

The project's honesty about its boundary—release stays manual—is rarer and more credible than the typical 'fully autonomous agent' pitch.

Concepts & terms
Verification harness
A system that runs an AI agent inside a controlled environment and checks its outputs against objective criteria, typically by executing real commands or validating artifacts, to ensure the agent actually did the work rather than just claiming it did.
Anchor state machine
A flow control structure where a task is broken into named stages (anchors), each with defined inputs, outputs, and verification gates. The engine advances from one anchor to the next only when all gates pass, creating an enforceable execution path.
Exit-code gating
Using the process exit code (0 for success, non-zero for failure) of real commands like tsc or eslint as the only acceptable proof that a verification step passed. This removes the model's subjective self-assessment from the loop.
Circuit breaker (agent)
A mechanism that stops an agent from retrying a failing stage indefinitely. In GateLoop, each anchor has a maxAttempts limit; exceeding it marks the anchor as escalated and halts the run, preventing wasted tokens on unproductive loops.
Data contract (emits/consumes)
An explicit declaration of what artifacts a stage produces (emits) and what inputs it requires (consumes). The engine enforces these contracts during verification, turning informal handoffs between stages into verifiable checks.
OnEscalate rollback
A flow-level rule that, when a downstream stage fails and escalates, instructs the engine to jump to a specific upstream stage (e.g., redo the design) instead of forcing through with a broken artifact.
Source: juejin.cn ↗ Google Translate ↗ Backup ↗