跪拜 Guibai
← Back to the summary

AI Agents Don't Need a Smarter Model—They Need a Runtime That Enforces 'Done'

Subtitle: GateLoop — a unified execution engine combining flow gating + multi-skill collaboration + exit-code verification

One-liner: The model does the work; the engine decides when "done" actually counts.


1. Stronger models, messier delivery

Start with three repeatedly proven facts:

Phenomenon Concrete manifestation Root cause
Agent says "verification passed" Actually never ran tsc; code with compile errors merged to main Steps are suggestions, no enforced execution point
Long-task state drift By round 40 of a conversation, model forgets which step it's on and what upstream produced State lives in the model's head
Infinite loop burns tokens Same error fixed 20 times; bill blows up before discovery No circuit breaker

The traditional fix is writing "please verify before committing" into the system prompt. The problem: these are suggestions to an LLM, not constraints. It can selectively obey, and selectively "pretend" to obey.

The industry context is now clear. In 2026, verification harnesses are recognized as the highest-leverage practice for unattended agent reliability, but a report that reviewed 21 agent harnesses gave a more direct conclusion:

"They all solve the problem of running an agent, none solves the problem of verifying an agent. None of them define 'done' as something you can check."

Models are getting stronger, context windows are growing, but the execution layer is unattended: who decides how far the process has gone, how stages hand off, what counts as done, and what happens on failure. This is what this article addresses—not another prompt trick, but a runtime that moves discipline from "model self-discipline" into "engine enforcement": GateLoop.

A real incident

Constraints placed in the wrong place—no matter how long the prompt—won't help. This can't be argued by citing others' failures, so we made it a reproducible scene left in the repo (demo/gateloop-story/): a sample file intentionally seeded with a TS type error, an ESLint violation, and a Prettier format violation, split into three anchors lint → ts → format, each guarded by the real tool's exit 0. Here is the actual record from the day this article was written (verbatim, unedited):

$ nut gateloop next anchor.config.json   # Only fixed format, type error remains

📌 Stage: ts (tsc pass)   🔁 Attempt: 1/3
❌ Verification: FAIL (1 check)
  ❌ Custom: npx tsc --noEmit --strict broken-sample.ts (5897ms)
     broken-sample.ts(4,9): error TS2322: Type 'number' is not assignable to type 'string'.
➡️ ❌ Verification not passed (1 error, 1/3 attempts). Fix then call next again.

The timeline fully reproduced the incident structure: the first eslint gate caught the format violation (Replace ',' with ';'); after formatting, the second tsc gate caught the type error TS2322 — this step is the crux: if verification relies on model self-assessment, "I think it's fixed" passes; wired into verify.custom, it's just one line of output when the engine re-runs the command and gets exit 1. The fix isn't making SKILL.md harsher, but hooking the verification command into the gate so the engine re-runs and judges.

The problem is where constraints live, not prompt length. This lesson runs through the article.


2. GateLoop: installing an execution runtime for agents

GateLoop is a runtime harness. A skill declares only "what to do" (What); the engine enforces "what counts as done" (How). It differs from prompt techniques in enforcement power: steps in SKILL.md are suggestions to the model; verify in anchor.config.json is a gate to the model.

Its capability surface is three lines, all implemented in real code in src/gateloop/:

① Flow gating   — anchor state machine advancement, each stage must pass its gate to move on (state.ts)
② Collaboration pipeline — emits/consumes data contracts between stages, multi-skill cross-archive context (flow.config)
③ Verification gate — before advancing, engine re-runs real commands; exit 0 counts (verify.ts)

Delivered as CLI and API (npm package ai-rail, released v0.0.0-beta.8, bin command nut):

# CLI: all flow operations are one command
nut gateloop init <manifest> --task "Implement user list page"
nut gateloop next <manifest>      # advance (internally forces re-verify + circuit breaker)
nut gateloop status <manifest>    # disconnect recovery: where are we now
nut gateloop retry / reset / check / gc / subtask-add / subtask-done / emit / mem-write / mem-read

# API: same capabilities, programmatic call
import { GateLoop, verify } from 'ai-rail/gateloop'
const loop = new GateLoop(loadManifest('./anchor.config.json'))
await loop.init('Implement user list page')   // → instruction: first do inventory (objective checklist)
await loop.next()                   // → engine re-runs verification, exit 0 to advance

A skill's "execution contract" looks like this (real config excerpt, frontend-codegen frontend code generation skill):

{
  "name": "frontend-codegen",
  "description": "Sole entry for frontend code generation (GateLoop harness) — single skill completes design→generation→quality check. v14 merged frontend-design/css-style-writer/code-quality-guard into inline rules (rules/*.mdc), eliminating multi-skill split mis-triggers",
  "scratchDir": ".specflow/scratch",
  "scratchGcHours": 72,
  "maxAttempts": 5,                    // circuit breaker: max 5 attempts per stage
  "anchors": [
    { "id": "inventory",               // stage 1: objective environment and style baseline, script-produced, model can't alter
      "name": "Environment and style baseline (objective)",
      "brief": "Script produces inventory.json: enumerates all .ts/.tsx files under src/ and flags line-count violations (objective baseline, model not involved in judgment). Fixed conventions (non-negotiable, enforced throughout): component form = <a-b>/index.tsx + <a-b>/index.scss; cyclomatic complexity ≤ 15……",
      "produces": [".specflow/context/inventory.json"],
      "emits": ["inventory"],
      "verify": {
        "produceScripts": [
          "node ${skillDir}/scripts/inventory.mjs --output .specflow/context/inventory.json --root src"
        ],
        "autoEmit": { "inventory": ".specflow/context/inventory.json" },  // script success automatically produces artifact; zero model involvement
        "modelWritable": false,        // inventory only script-written; engine re-runs and overwrites at verify, prevents forgery
        "producesFresh": true          // artifact must be newer than this attempt; reusing old files fails
      }
    },
    { "id": "design",                  // stage 2: design thinking in four steps, must cover all inventory violations
      "name": "Design thinking: impact scope→approach→changes→task list",
      "reads": ["rules/design-rules.mdc"],   // rules inlined by engine, no Step 0 needed
      "consumes": ["inventory"],
      "produces": [".specflow/context/current.json"],
      "emits": ["frontendPlan", "architectureReview"],
      "verify": {
        "schema": { "type": "object",
          "required": ["frontendPlan", "architectureReview"],
          "properties": {
            "frontendPlan": { "required": ["impactScope", "changes", "tasks", "functionOwnership", "stateClassification", "efficiencyAudit", "moduleBoundary", "edgeCases", "errorStates", "coverage"] },
            "architectureReview": { "required": ["verdicts"] }   // every verdict must carry metric/fact/verdict/reason
          } },
        "producesFresh": true,
        "custom": [
          "node ${skillDir}/scripts/verify-coverage.mjs --inventory .specflow/context/inventory.json --plan .specflow/context/current.json",
          "npx nut gateloop metric design --input .specflow/context/current.json",
          "npx nut gateloop metric architecture --input .specflow/context/current.json"
        ],
        "review": { "output": ".specflow/context/design-review.json", "requireEvidence": true,
          "checklist": [
            { "id": "impact-scope",       "rule": "Impact scope complete: all modules/files/dependency chains affected by changes identified, no missing cascading changes", "mode": "model" },
            { "id": "changes-convention", "rule": "Change list conforms to fixed conventions: component path <a-b>/index.tsx + <a-b>/index.scss (kebab-case directory)", "mode": "model" },
            { "id": "tasks-derivation",   "rule": "Task list derived from changes: no task missing files, no file missing task", "mode": "model" },
            /* ……9 items total: layout decisions/module boundaries/state classification/function ownership/coupling health/efficiency audit, each with verdict + line-numbered evidence */
          ] }
      }
    },
    { "id": "code",                    // stage 3: generate per task + quality acceptance, exit 0 to pass
      "name": "Generate per task + quality acceptance",
      "reads": ["RULE.mdc", "rules/style-rules.mdc", "rules/quality-rules.mdc"],
      "consumes": ["frontendPlan", "architectureReview"],
      "requiresSubtasks": true,        // execute per frontendPlan.tasks, all green to advance
      "maxAttempts": 8,
      "produces": ["src/**/*.tsx", "src/**/*.ts", "src/**/*.scss"],
      "verify": {
        "tsc": true, "build": true, "producesFresh": true,
        "custom": [
          "npx nut gateloop metric style-crosscheck --dir src/",
          "npx nut gateloop metric code --dir src/",
          "node ${skillDir}/scripts/verify-completion.mjs --root src --plan .specflow/context/current.json"
        ],
        "review": { "output": ".specflow/context/code-review.json", "requireEvidence": true,
          "checklist": [
            { "id": "component-convention", "rule": "Component form conforms to fixed conventions: <a-b>/index.tsx + <a-b>/index.scss (entry index, fixed)", "mode": "model" },
            { "id": "data-flow",            "rule": "Data flow clear: props injected one-way, state classification per frontendPlan.stateClassification", "mode": "model" },
            { "id": "style-state-coverage", "rule": "Full state coverage: hover/focus/disabled + loading/empty/error all covered", "mode": "model" }
            /* ……11 items, each with verdict + line-numbered evidence */
          ] }
      }
    }
  ]
}

Mechanisms behind several fields: modelWritable: false means the inventory is script-decided—when verifying, the engine re-runs produceScripts and overwrites the artifact; model forgery is futile. producesFresh means the artifact must be newer than this attempt; passing off old files fails. autoEmit makes inventory zero-model-involvement: script in, artifact out. reads lets the engine inline rules, no longer relying on the model to read them.


3. Gating: deterministic goes to scripts, understanding goes to checklists

The verification layer's core is a 5-way division of labor—put each quality requirement in its proper place without overstepping:

A Convergence (compiles / no any / cyclomatic complexity) → exit code    ← script, 100% deterministic
B Completeness (fields present)                 → schema        ← script blocks "did you think of everything"
C Metrics (dependency cycles / hotspots)        → script coarse filter → model fine judgment → feed back
D Subjective (is the split reasonable)          → review's four gates ← model judges + script checks form
E Pure aesthetics                               → not in verify

Practical meaning of this boundary: you won't stuff "is the layout reasonable" into tsc, and you won't hand "does a dependency cycle exist" to model self-assessment—the former can't be measured, the latter can't be measured accurately. C-class is a mix of both: script first runs dependency graph finding 12 suspicious cycles, model then judges which are real cycles and which are type-reference false positives, conclusion fed back as artifact for downstream consumption.

D-class (subjective) is the most overlooked and most often problematic. Model self-review lenient, so review has four gates:

  1. Completeness: every checklist item must have a verdict; missing review fails
  2. Anti-forgery: for items with mode: 'script', engine re-runs the command itself and checks against model's written conclusion—model says "similarity 98%", engine runs similarity script, number mismatch fails directly
  3. Evidence: each evidence must carry real references (file + line), empty talk doesn't pass
  4. Fail must change: any verdict fail that is blocking fails the whole stage

This is the "two paths for detection checklist" in practice: from the user-supplied detection checklist, anything scriptable gets 100% exit-code judgment; what needs understanding, the model goes through once and checks item by item with traces left.


4. Flow: state machine, contracts, circuit breaker

The engine's advance heart is doNext(), gate order deliberately designed (state.ts real implementation):

call next()
 ├─ [Gate1] requiresSubtasks? → subtasks not all green → reject advance
 ├─ force re-run verify (real commands, exit-code judge)
 ├─ [Gate2] emitted artifacts declared exist?
 ├─ [Gate3] writesMemory declared decision memory written back?
 ├─ attempts++
 ├─ verification failed?
 │    ├─ attempts >= maxAttempts → circuit breaker (escalated)
 │    └─ else → return error, let model fix and retry
 └─ verification passed? → advance to next anchor

Three easily underestimated details:

State's sole writer. anchor-state.json can only be written internally by the engine; not exported to model. Model can read state, request changes via API, but cannot modify the file itself. After disconnect, nut gateloop status always matches true progress. When state file is corrupted (process force-killed), engine backs up the original to .corrupt-<timestamp> then re-initializes—progress isn't silently lost.

Data contracts. Stages hand off via consumes / emits explicitly: declared emits: ["frontendPlan"] but not produced, verification fails. Data flow goes from "implicit convention" to "explicit contract"; upstream output auto-inlined by engine into next stage's instructions—model doesn't need to find it, can't grab wrong one.

Circuit breaker and cycle detection. Each stage has independent maxAttempts; consecutive failures exceed limit and enter escalated shutdown, no more token burn. Flow layer also has maxJumps cycle detection: onPass jumps forming a loop (A→B→A) get detected and shutdown, not infinite idling.


5. Collaboration: multi-skill orchestration and information pipeline

Single skill uses anchors for internal management; multiple skills relay with flow for global management (flow.config.json):

{
  "name": "full-stack-deliver",
  "flow": [
    { "id": "design", "delegateSkill": "web-ui-design",
      "writesMemory": ["designDecision"], "readsContext": ["#req"] },
    { "id": "code",   "delegateSkill": "frontend-codegen",
      "readsContext": ["design"],
      "onEscalate": { "jumpTo": "design" } }      // design not up to standard → rollback to requirements stage redo
  ]
}

Long-chain collaboration's biggest enemy is information decay: first skill's conclusion reaches last as one sentence. GateLoop fights it with three pipelines:

In the real repo, this pattern runs on multiple skills: frontend-codegen (inventory → design → code three anchors), pvm (detect → install → execute), code-env (env detect → pull code create branch → install runtime → install deps → startup verify, five anchors). Different domains, identical gate pattern—this is "declaration is capability": skill author writes only checklist and verification conditions; state machine, circuit breaker, isolation, resume provided uniformly by engine.


6. Why would the model obey you?

A question that must be answered head-on: all this assumes the model is willing to call nut gateloop next. If the model doesn't call, or directly lies saying "done", the engine never gets a chance to run—who guarantees it follows the process?

First admit: no 100% guarantee. A model can refuse to call, can lie. Any solution claiming to "guarantee model execution" can't do it.

But note—we don't guarantee "model execution", we guarantee "no real execution = zero advancement". These are completely different:

Path A: real execution     → init → produce files → next (engine forces re-run exit code) → state advances
Path B: lie saying done    → says flowery things in conversation
                          but anchor-state.json hasn't moved a step, artifacts don't exist, mtime unchanged
                          no objective evidence any third party can check

The real constraint isn't "can you lie", but "what does lying get you"—answer: lies cannot produce advancement. State file is engine-only written; artifact existence is a physical fact on disk; downstream inlined data is injected only after true advancement. "Cutting corners" path is exactly the "correct execution" path; lying is mechanically a dead end.

Four defense layers stacked: entry layer (skill iron law requires init/next) → verification layer (next forces re-run real commands) → anti-forgery layer (review's script items re-run by engine) → audit layer (state file + logs + artifact timestamps, checkable anytime).

One sentence: we don't bet on model honesty, we bet on "verification is deterministic". Banks don't guarantee no one forges bills, but guarantee fakes can't pass the counterfeit detector. Engine is that detector—don't bet on human nature, bet on machine.


7. Experiments: six reproducible comparisons

Terminal outputs below are from actual measurement records during engine development (fixtures kept in repo git history, demos/gateloop/ directory later removed during cleanup).

Experiment 1: It says verified, the gate doesn't accept

code anchor declares "verify": { "tsc": true }. Code deliberately leaves a compile error; model calls next:

$ nut gateloop next anchor.config.json

📌 Stage: code (coding)   🔁 Attempt: 1/5
🔍 Verification: tsc --noEmit (executed internally by engine)

❌ Verification: FAIL (2 checks)
  ✅ Artifact existence (6ms)
  ❌ tsc (2508ms)
     src/utils.ts(4,14): error TS2322: Type 'string' is not assignable to type 'number'.

➡️ ❌ Verification not passed (1 error, 1/5 attempts). Fix then call next again.

Model can claim "tsc passed" a hundred times, but engine runs it itself before advancing. To pass, you must actually fix the code.

Experiment 2: Infinite loop circuit-broken

verify.custom constantly node -e "process.exit(1)", maxAttempts: 2:

$ nut gateloop next anchor.config.json
📌 Stage: build (build)   🔁 Attempt: 1/2
❌ Verification: FAIL (1 check)
  ❌ Custom: node -e "process.exit(1)" (154ms)
➡️ ❌ Verification not passed (1 error, 1/2 attempts). Fix then call next again.

$ nut gateloop next anchor.config.json
📌 Stage: build (build)   🔁 Attempt: 2/2
🛑 Circuit broken: consecutive failures exceed limit, need human intervention or fallback.

Second attempt still fails; state machine marks anchor escalated, auto-stops. No meaningless token burn.

Experiment 3: Doing nothing and trying to hand in

scan anchor declares artifact todo-report.json. Model does nothing and calls next:

$ nut gateloop next anchor.config.json
📌 Stage: scan (scan and summarize TODO)   🔁 Attempt: 1/3
❌ Verification: FAIL (1 check)
  ❌ Artifact existence (7ms)
     Missing: todo-report.json
➡️ ❌ Verification not passed (1 error, 1/3 attempts). Fix then call next again.

Without GateLoop, model would just say "I scanned, no TODO" and hand in; with GateLoop, it must actually produce artifact file.

Experiments 4-5: Stability and completeness (mechanism verification)

Experiment 6: Real file, real tools, real fix

Give a .ts file seeded with lint / TS type / formatting issues, do two skills (fully reproducible demo: demo/gateloop-story/, run instructions in anchor.config.json three anchors):

{ "anchors": [
  { "id": "lint",   "verify": { "custom": "npx eslint ../broken-sample.ts" } },
  { "id": "ts",     "verify": { "custom": "npx tsc --noEmit --strict ... ../broken-sample.ts" } },
  { "id": "format", "verify": { "custom": "npx prettier --check ../broken-sample.ts" } }
] }

Real run: first next blocked by lint gate (19 errors, attempt 1/3, no advance); after each type fixed, call next again, outputs sequentially ✅ Verification passed, advanced to ts stage → advanced to format stage → ✅ all stages completed.

Difference: bare skill defines "fixed" as model's statement; GateLoop skill defines it as three tools' exit 0. Missing one gate isn't done.


8. Boundary: what to automate, what to leave to humans

Finally, the boundary. GateLoop's automatic closed loop only covers development + testing, stops at release:

Scripts (deterministic) → exit code gates, automatic — verify it runs
Model (understanding)   → write code, do design, automatic — turn requirements into code
Human (irreversible)    → release to production, manual      — only human knows "is it right"

"Is production right" scripts can't verify; leave to human experience. Daring to say "we don't manage that" is more credible than claiming "fully automatic".


Repository

Source, docs, Issues all here, open source free, welcome. If useful, give a star.


npm package

Appendix: Implementation shape

Engine provided as npm package ai-rail (released v0.0.0-beta.8, bin command nut), pure CLI + API forms. Internal state and workspace functions not exported outward—all capabilities wrapped through GateLoop class, preventing "bypass engine to directly change state" from API level. verify command-class validators are tsc / build / test / custom (any real command can go into custom), plus full gate set schema / producesExist / producesFresh / review / produceScripts.

Engine source (src/gateloop/) and companion skill (skill-creator-gateloop) are in repo directly readable and runnable.


If you're also building AI coding agents, consider pulling "discipline" out of prompts and giving it to the engine. The model does the work; the engine decides when "done" counts.