AI Agents Don't Need a Smarter Model—They Need a Runtime That Enforces 'Done'
Subtitle: GateLoop — a unified execution engine combining flow gating + multi-skill collaboration + exit-code verification
One-liner: The model does the work; the engine decides when "done" actually counts.
1. Stronger models, messier delivery
Start with three repeatedly proven facts:
| Phenomenon | Concrete manifestation | Root cause |
|---|---|---|
| Agent says "verification passed" | Actually never ran tsc; code with compile errors merged to main | Steps are suggestions, no enforced execution point |
| Long-task state drift | By round 40 of a conversation, model forgets which step it's on and what upstream produced | State lives in the model's head |
| Infinite loop burns tokens | Same error fixed 20 times; bill blows up before discovery | No circuit breaker |
The traditional fix is writing "please verify before committing" into the system prompt. The problem: these are suggestions to an LLM, not constraints. It can selectively obey, and selectively "pretend" to obey.
The industry context is now clear. In 2026, verification harnesses are recognized as the highest-leverage practice for unattended agent reliability, but a report that reviewed 21 agent harnesses gave a more direct conclusion:
"They all solve the problem of running an agent, none solves the problem of verifying an agent. None of them define 'done' as something you can check."
Models are getting stronger, context windows are growing, but the execution layer is unattended: who decides how far the process has gone, how stages hand off, what counts as done, and what happens on failure. This is what this article addresses—not another prompt trick, but a runtime that moves discipline from "model self-discipline" into "engine enforcement": GateLoop.
A real incident
Constraints placed in the wrong place—no matter how long the prompt—won't help. This can't be argued by citing others' failures, so we made it a reproducible scene left in the repo (demo/gateloop-story/): a sample file intentionally seeded with a TS type error, an ESLint violation, and a Prettier format violation, split into three anchors lint → ts → format, each guarded by the real tool's exit 0. Here is the actual record from the day this article was written (verbatim, unedited):
$ nut gateloop next anchor.config.json # Only fixed format, type error remains
📌 Stage: ts (tsc pass) 🔁 Attempt: 1/3
❌ Verification: FAIL (1 check)
❌ Custom: npx tsc --noEmit --strict broken-sample.ts (5897ms)
broken-sample.ts(4,9): error TS2322: Type 'number' is not assignable to type 'string'.
➡️ ❌ Verification not passed (1 error, 1/3 attempts). Fix then call next again.
The timeline fully reproduced the incident structure: the first eslint gate caught the format violation (Replace ',' with ';'); after formatting, the second tsc gate caught the type error TS2322 — this step is the crux: if verification relies on model self-assessment, "I think it's fixed" passes; wired into verify.custom, it's just one line of output when the engine re-runs the command and gets exit 1. The fix isn't making SKILL.md harsher, but hooking the verification command into the gate so the engine re-runs and judges.
The problem is where constraints live, not prompt length. This lesson runs through the article.
2. GateLoop: installing an execution runtime for agents
GateLoop is a runtime harness. A skill declares only "what to do" (What); the engine enforces "what counts as done" (How). It differs from prompt techniques in enforcement power: steps in SKILL.md are suggestions to the model; verify in anchor.config.json is a gate to the model.
Its capability surface is three lines, all implemented in real code in src/gateloop/:
① Flow gating — anchor state machine advancement, each stage must pass its gate to move on (state.ts)
② Collaboration pipeline — emits/consumes data contracts between stages, multi-skill cross-archive context (flow.config)
③ Verification gate — before advancing, engine re-runs real commands; exit 0 counts (verify.ts)
Delivered as CLI and API (npm package ai-rail, released v0.0.0-beta.8, bin command nut):
# CLI: all flow operations are one command
nut gateloop init <manifest> --task "Implement user list page"
nut gateloop next <manifest> # advance (internally forces re-verify + circuit breaker)
nut gateloop status <manifest> # disconnect recovery: where are we now
nut gateloop retry / reset / check / gc / subtask-add / subtask-done / emit / mem-write / mem-read
# API: same capabilities, programmatic call
import { GateLoop, verify } from 'ai-rail/gateloop'
const loop = new GateLoop(loadManifest('./anchor.config.json'))
await loop.init('Implement user list page') // → instruction: first do inventory (objective checklist)
await loop.next() // → engine re-runs verification, exit 0 to advance
A skill's "execution contract" looks like this (real config excerpt, frontend-codegen frontend code generation skill):
{
"name": "frontend-codegen",
"description": "Sole entry for frontend code generation (GateLoop harness) — single skill completes design→generation→quality check. v14 merged frontend-design/css-style-writer/code-quality-guard into inline rules (rules/*.mdc), eliminating multi-skill split mis-triggers",
"scratchDir": ".specflow/scratch",
"scratchGcHours": 72,
"maxAttempts": 5, // circuit breaker: max 5 attempts per stage
"anchors": [
{ "id": "inventory", // stage 1: objective environment and style baseline, script-produced, model can't alter
"name": "Environment and style baseline (objective)",
"brief": "Script produces inventory.json: enumerates all .ts/.tsx files under src/ and flags line-count violations (objective baseline, model not involved in judgment). Fixed conventions (non-negotiable, enforced throughout): component form = <a-b>/index.tsx + <a-b>/index.scss; cyclomatic complexity ≤ 15……",
"produces": [".specflow/context/inventory.json"],
"emits": ["inventory"],
"verify": {
"produceScripts": [
"node ${skillDir}/scripts/inventory.mjs --output .specflow/context/inventory.json --root src"
],
"autoEmit": { "inventory": ".specflow/context/inventory.json" }, // script success automatically produces artifact; zero model involvement
"modelWritable": false, // inventory only script-written; engine re-runs and overwrites at verify, prevents forgery
"producesFresh": true // artifact must be newer than this attempt; reusing old files fails
}
},
{ "id": "design", // stage 2: design thinking in four steps, must cover all inventory violations
"name": "Design thinking: impact scope→approach→changes→task list",
"reads": ["rules/design-rules.mdc"], // rules inlined by engine, no Step 0 needed
"consumes": ["inventory"],
"produces": [".specflow/context/current.json"],
"emits": ["frontendPlan", "architectureReview"],
"verify": {
"schema": { "type": "object",
"required": ["frontendPlan", "architectureReview"],
"properties": {
"frontendPlan": { "required": ["impactScope", "changes", "tasks", "functionOwnership", "stateClassification", "efficiencyAudit", "moduleBoundary", "edgeCases", "errorStates", "coverage"] },
"architectureReview": { "required": ["verdicts"] } // every verdict must carry metric/fact/verdict/reason
} },
"producesFresh": true,
"custom": [
"node ${skillDir}/scripts/verify-coverage.mjs --inventory .specflow/context/inventory.json --plan .specflow/context/current.json",
"npx nut gateloop metric design --input .specflow/context/current.json",
"npx nut gateloop metric architecture --input .specflow/context/current.json"
],
"review": { "output": ".specflow/context/design-review.json", "requireEvidence": true,
"checklist": [
{ "id": "impact-scope", "rule": "Impact scope complete: all modules/files/dependency chains affected by changes identified, no missing cascading changes", "mode": "model" },
{ "id": "changes-convention", "rule": "Change list conforms to fixed conventions: component path <a-b>/index.tsx + <a-b>/index.scss (kebab-case directory)", "mode": "model" },
{ "id": "tasks-derivation", "rule": "Task list derived from changes: no task missing files, no file missing task", "mode": "model" },
/* ……9 items total: layout decisions/module boundaries/state classification/function ownership/coupling health/efficiency audit, each with verdict + line-numbered evidence */
] }
}
},
{ "id": "code", // stage 3: generate per task + quality acceptance, exit 0 to pass
"name": "Generate per task + quality acceptance",
"reads": ["RULE.mdc", "rules/style-rules.mdc", "rules/quality-rules.mdc"],
"consumes": ["frontendPlan", "architectureReview"],
"requiresSubtasks": true, // execute per frontendPlan.tasks, all green to advance
"maxAttempts": 8,
"produces": ["src/**/*.tsx", "src/**/*.ts", "src/**/*.scss"],
"verify": {
"tsc": true, "build": true, "producesFresh": true,
"custom": [
"npx nut gateloop metric style-crosscheck --dir src/",
"npx nut gateloop metric code --dir src/",
"node ${skillDir}/scripts/verify-completion.mjs --root src --plan .specflow/context/current.json"
],
"review": { "output": ".specflow/context/code-review.json", "requireEvidence": true,
"checklist": [
{ "id": "component-convention", "rule": "Component form conforms to fixed conventions: <a-b>/index.tsx + <a-b>/index.scss (entry index, fixed)", "mode": "model" },
{ "id": "data-flow", "rule": "Data flow clear: props injected one-way, state classification per frontendPlan.stateClassification", "mode": "model" },
{ "id": "style-state-coverage", "rule": "Full state coverage: hover/focus/disabled + loading/empty/error all covered", "mode": "model" }
/* ……11 items, each with verdict + line-numbered evidence */
] }
}
}
]
}
Mechanisms behind several fields: modelWritable: false means the inventory is script-decided—when verifying, the engine re-runs produceScripts and overwrites the artifact; model forgery is futile. producesFresh means the artifact must be newer than this attempt; passing off old files fails. autoEmit makes inventory zero-model-involvement: script in, artifact out. reads lets the engine inline rules, no longer relying on the model to read them.
3. Gating: deterministic goes to scripts, understanding goes to checklists
The verification layer's core is a 5-way division of labor—put each quality requirement in its proper place without overstepping:
A Convergence (compiles / no any / cyclomatic complexity) → exit code ← script, 100% deterministic
B Completeness (fields present) → schema ← script blocks "did you think of everything"
C Metrics (dependency cycles / hotspots) → script coarse filter → model fine judgment → feed back
D Subjective (is the split reasonable) → review's four gates ← model judges + script checks form
E Pure aesthetics → not in verify
Practical meaning of this boundary: you won't stuff "is the layout reasonable" into tsc, and you won't hand "does a dependency cycle exist" to model self-assessment—the former can't be measured, the latter can't be measured accurately. C-class is a mix of both: script first runs dependency graph finding 12 suspicious cycles, model then judges which are real cycles and which are type-reference false positives, conclusion fed back as artifact for downstream consumption.
D-class (subjective) is the most overlooked and most often problematic. Model self-review lenient, so review has four gates:
- Completeness: every checklist item must have a verdict; missing review fails
- Anti-forgery: for items with
mode: 'script', engine re-runs the command itself and checks against model's written conclusion—model says "similarity 98%", engine runs similarity script, number mismatch fails directly - Evidence: each evidence must carry real references (file + line), empty talk doesn't pass
- Fail must change: any verdict fail that is blocking fails the whole stage
This is the "two paths for detection checklist" in practice: from the user-supplied detection checklist, anything scriptable gets 100% exit-code judgment; what needs understanding, the model goes through once and checks item by item with traces left.
4. Flow: state machine, contracts, circuit breaker
The engine's advance heart is doNext(), gate order deliberately designed (state.ts real implementation):
call next()
├─ [Gate1] requiresSubtasks? → subtasks not all green → reject advance
├─ force re-run verify (real commands, exit-code judge)
├─ [Gate2] emitted artifacts declared exist?
├─ [Gate3] writesMemory declared decision memory written back?
├─ attempts++
├─ verification failed?
│ ├─ attempts >= maxAttempts → circuit breaker (escalated)
│ └─ else → return error, let model fix and retry
└─ verification passed? → advance to next anchor
Three easily underestimated details:
State's sole writer. anchor-state.json can only be written internally by the engine; not exported to model. Model can read state, request changes via API, but cannot modify the file itself. After disconnect, nut gateloop status always matches true progress. When state file is corrupted (process force-killed), engine backs up the original to .corrupt-<timestamp> then re-initializes—progress isn't silently lost.
Data contracts. Stages hand off via consumes / emits explicitly: declared emits: ["frontendPlan"] but not produced, verification fails. Data flow goes from "implicit convention" to "explicit contract"; upstream output auto-inlined by engine into next stage's instructions—model doesn't need to find it, can't grab wrong one.
Circuit breaker and cycle detection. Each stage has independent maxAttempts; consecutive failures exceed limit and enter escalated shutdown, no more token burn. Flow layer also has maxJumps cycle detection: onPass jumps forming a loop (A→B→A) get detected and shutdown, not infinite idling.
5. Collaboration: multi-skill orchestration and information pipeline
Single skill uses anchors for internal management; multiple skills relay with flow for global management (flow.config.json):
{
"name": "full-stack-deliver",
"flow": [
{ "id": "design", "delegateSkill": "web-ui-design",
"writesMemory": ["designDecision"], "readsContext": ["#req"] },
{ "id": "code", "delegateSkill": "frontend-codegen",
"readsContext": ["design"],
"onEscalate": { "jumpTo": "design" } } // design not up to standard → rollback to requirements stage redo
]
}
Long-chain collaboration's biggest enemy is information decay: first skill's conclusion reaches last as one sentence. GateLoop fights it with three pipelines:
- readsContext context archive: upstream stage ends, engine auto-archives decisions and artifacts, downstream skill pulls by key, not model retelling
- writesMemory decision memory: key decisions (choices, conventions) written to disk as deterministic source of truth, queryable across tasks and rounds
- onEscalate rollback: downstream verification finds upstream design problematic, jumps back to upstream redo, instead of forcing through with errors
In the real repo, this pattern runs on multiple skills: frontend-codegen (inventory → design → code three anchors), pvm (detect → install → execute), code-env (env detect → pull code create branch → install runtime → install deps → startup verify, five anchors). Different domains, identical gate pattern—this is "declaration is capability": skill author writes only checklist and verification conditions; state machine, circuit breaker, isolation, resume provided uniformly by engine.
6. Why would the model obey you?
A question that must be answered head-on: all this assumes the model is willing to call nut gateloop next. If the model doesn't call, or directly lies saying "done", the engine never gets a chance to run—who guarantees it follows the process?
First admit: no 100% guarantee. A model can refuse to call, can lie. Any solution claiming to "guarantee model execution" can't do it.
But note—we don't guarantee "model execution", we guarantee "no real execution = zero advancement". These are completely different:
Path A: real execution → init → produce files → next (engine forces re-run exit code) → state advances
Path B: lie saying done → says flowery things in conversation
but anchor-state.json hasn't moved a step, artifacts don't exist, mtime unchanged
no objective evidence any third party can check
The real constraint isn't "can you lie", but "what does lying get you"—answer: lies cannot produce advancement. State file is engine-only written; artifact existence is a physical fact on disk; downstream inlined data is injected only after true advancement. "Cutting corners" path is exactly the "correct execution" path; lying is mechanically a dead end.
Four defense layers stacked: entry layer (skill iron law requires init/next) → verification layer (next forces re-run real commands) → anti-forgery layer (review's script items re-run by engine) → audit layer (state file + logs + artifact timestamps, checkable anytime).
One sentence: we don't bet on model honesty, we bet on "verification is deterministic". Banks don't guarantee no one forges bills, but guarantee fakes can't pass the counterfeit detector. Engine is that detector—don't bet on human nature, bet on machine.
7. Experiments: six reproducible comparisons
Terminal outputs below are from actual measurement records during engine development (fixtures kept in repo git history, demos/gateloop/ directory later removed during cleanup).
Experiment 1: It says verified, the gate doesn't accept
code anchor declares "verify": { "tsc": true }. Code deliberately leaves a compile error; model calls next:
$ nut gateloop next anchor.config.json
📌 Stage: code (coding) 🔁 Attempt: 1/5
🔍 Verification: tsc --noEmit (executed internally by engine)
❌ Verification: FAIL (2 checks)
✅ Artifact existence (6ms)
❌ tsc (2508ms)
src/utils.ts(4,14): error TS2322: Type 'string' is not assignable to type 'number'.
➡️ ❌ Verification not passed (1 error, 1/5 attempts). Fix then call next again.
Model can claim "tsc passed" a hundred times, but engine runs it itself before advancing. To pass, you must actually fix the code.
Experiment 2: Infinite loop circuit-broken
verify.custom constantly node -e "process.exit(1)", maxAttempts: 2:
$ nut gateloop next anchor.config.json
📌 Stage: build (build) 🔁 Attempt: 1/2
❌ Verification: FAIL (1 check)
❌ Custom: node -e "process.exit(1)" (154ms)
➡️ ❌ Verification not passed (1 error, 1/2 attempts). Fix then call next again.
$ nut gateloop next anchor.config.json
📌 Stage: build (build) 🔁 Attempt: 2/2
🛑 Circuit broken: consecutive failures exceed limit, need human intervention or fallback.
Second attempt still fails; state machine marks anchor escalated, auto-stops. No meaningless token burn.
Experiment 3: Doing nothing and trying to hand in
scan anchor declares artifact todo-report.json. Model does nothing and calls next:
$ nut gateloop next anchor.config.json
📌 Stage: scan (scan and summarize TODO) 🔁 Attempt: 1/3
❌ Verification: FAIL (1 check)
❌ Artifact existence (7ms)
Missing: todo-report.json
➡️ ❌ Verification not passed (1 error, 1/3 attempts). Fix then call next again.
Without GateLoop, model would just say "I scanned, no TODO" and hand in; with GateLoop, it must actually produce artifact file.
Experiments 4-5: Stability and completeness (mechanism verification)
- Multiple generation of same problem: same error code repeatedly
nut gateloop check3 times, output except time byte-identical—determinism comes from pipeline, not model's "improvisation". - Subtask closure: stages with
requiresSubtasks: true, if one subtask not done,nextalways blocked until all done.
Experiment 6: Real file, real tools, real fix
Give a .ts file seeded with lint / TS type / formatting issues, do two skills (fully reproducible demo: demo/gateloop-story/, run instructions in anchor.config.json three anchors):
- Bare skill (only SKILL.md, no manifest): after fixing per docs declares "all fixed", external
npx eslintcatches 1 missed error—it will never discover it itself, all by conscience - GateLoop skill: three types of issues split into three anchors, each guarded by real tool exit 0:
{ "anchors": [
{ "id": "lint", "verify": { "custom": "npx eslint ../broken-sample.ts" } },
{ "id": "ts", "verify": { "custom": "npx tsc --noEmit --strict ... ../broken-sample.ts" } },
{ "id": "format", "verify": { "custom": "npx prettier --check ../broken-sample.ts" } }
] }
Real run: first next blocked by lint gate (19 errors, attempt 1/3, no advance); after each type fixed, call next again, outputs sequentially ✅ Verification passed, advanced to ts stage → advanced to format stage → ✅ all stages completed.
Difference: bare skill defines "fixed" as model's statement; GateLoop skill defines it as three tools' exit 0. Missing one gate isn't done.
8. Boundary: what to automate, what to leave to humans
Finally, the boundary. GateLoop's automatic closed loop only covers development + testing, stops at release:
Scripts (deterministic) → exit code gates, automatic — verify it runs
Model (understanding) → write code, do design, automatic — turn requirements into code
Human (irreversible) → release to production, manual — only human knows "is it right"
"Is production right" scripts can't verify; leave to human experience. Daring to say "we don't manage that" is more credible than claiming "fully automatic".
Repository
Source, docs, Issues all here, open source free, welcome. If useful, give a star.
npm package
- pnpm: pnpm add -D gateloop
- npm: npm i -D gateloop
Appendix: Implementation shape
Engine provided as npm package ai-rail (released v0.0.0-beta.8, bin command nut), pure CLI + API forms. Internal state and workspace functions not exported outward—all capabilities wrapped through GateLoop class, preventing "bypass engine to directly change state" from API level. verify command-class validators are tsc / build / test / custom (any real command can go into custom), plus full gate set schema / producesExist / producesFresh / review / produceScripts.
Engine source (src/gateloop/) and companion skill (skill-creator-gateloop) are in repo directly readable and runnable.
If you're also building AI coding agents, consider pulling "discipline" out of prompts and giving it to the engine. The model does the work; the engine decides when "done" counts.