跪拜 Guibai
← Back to the summary

Loop Engineering: The End of Prompting and the Rise of Autonomous Agent Systems

1. What is Loop Engineering

1.1 The Tipping Point: Two Sentences, Millions of Reposts

In early June 2026, the AI programming community was ignited by two sentences:

Peter Steinberger (Founder of OpenClaw): "You shouldn't be prompting coding agents anymore. You should be designing loops that prompt your agents."

Boris Cherny (Head of Anthropic Claude Code): "I don't prompt Claude anymore. I have loops running that prompt Claude and figuring out what to do. My job is to write loops."

These two sentences garnered over 5 million views on X (Twitter). Subsequently, Google engineer Addy Osmani published a blog post formally naming this concept Loop Engineering and providing a clear definitional framework. Developer Cobus Greyling then released the open-source reference implementation loop-engineering (MIT license, 7K+ Stars), turning the idea into a directly implementable toolset.

So, what exactly is Loop Engineering?

1.2 Core Definition

Loop Engineering is a system design methodology: you no longer write prompts for an AI Agent yourself, but instead design an automated loop system that lets the system discover work, assign tasks, verify results, persist state, and hand over to a human when necessary.

In Addy Osmani's words:

Loop engineering is replacing yourself as the person who prompts the agent. You design the system that does it instead.

The leverage point has shifted from "polishing a single prompt" to "designing the control system that orchestrates the Agent."

1.3 Four Paradigm Shifts in AI Engineering

Over the past two years, the focus of AI development has undergone four clear shifts:

Stage Core Question Human Role Typical Product
Prompt Engineering How to ask the AI? Prompt Craftsman Carefully polished prompt templates
Context Engineering What information to give the AI? Context Organizer AGENTS.md, CLAUDE.md, rule files
Harness Engineering How to organize AI's capabilities? Environment Builder Toolchains, MCP connectors, permission configs
Loop Engineering How to make AI continuously create results? System Architect Automated loops, state files, verification chains

With each shift, the human's role moves upward: from "operator" to "supervisor" to "architect." Loop Engineering is the current endpoint of this evolutionary path—you are no longer the person sitting in front of a chat box continuously inputting commands; you are the person designing the automated loop structure.

Image

1.4 Agent Loop vs Loop Engineering

Many people confuse Agent Loop and Loop Engineering—thinking that typing /loop 1d in the terminal counts as "doing loop engineering." That's not the case:

Dimension Agent Loop Loop Engineering
Essence Runtime mechanism / Product feature System design / Engineering methodology
What you do Start a repeating Agent task Design a complete system of discovery→execution→verification→handover
Granularity One recursive goal + scheduling Patterns, skills, state schema, security policies, cost models
Success Criterion "It's running again" "It runs correctly, cost-effectively, stops on failure, and its actions are understandable to humans"
Typical Product A single command like /loop 1d ... STATE.md + Skills + Worktree strategy + Verifier + Checklist

An analogy: Agent Loop is like while (!done) { agent.run(); }—the loop statement itself. Loop Engineering is like writing the entire main(): where does input come from, where is state stored, who writes and who verifies, what happens on timeout, where are logs written, when to break and call a human.

1.5 Three-Layer Conceptual Relationship

The concepts documentation of the reference library clarifies the layers:

Harness = The environment for a single Agent run (tools, permissions, rules)
Loop    = Harness + Scheduling + State + Verification Chain
Loop Engineering = The engineering practice of designing and operating the above Loop system
Concept Focus Analogy
Agent Harness Engineering What an Agent can use and know in a single session A single workstation's toolbox
Agent Loop Making the Agent run repeatedly at a rhythm The operation of a conveyor belt
Loop Engineering How the entire production line discovers tasks, divides labor, inspects quality, and hands over Factory design and SOPs

Image

2. The Six Building Blocks of Loop Engineering

A Loop that can truly run "unattended" is not a long prompt, but a system of six parts working in precise collaboration.

Image

2.1 Automations / Scheduling — The Heartbeat

Without scheduling, you only have a one-off Agent run. Scheduling is the heartbeat of the Loop.

Common implementation methods: /loop (Grok, Claude Code) — timed repeated execution; /schedule — trigger at specific times; GitHub Actions cron — team-level scheduled tasks; /goal — run until a verifiable condition is true; Custom harness scheduler.

Key attributes: Interval, whether to trigger immediately, single or repeating, whether to persist (survive restarts).

2.2 Worktrees — Safe Parallelism

When two Agents edit the same file simultaneously, you get merge hell. Git worktrees (or equivalent isolated checkouts) give each Agent its own working directory—shared history but not shared working tree.

In Grok: pass isolation: "worktree" to child Agents. In Claude Code: use --worktree or dedicated sessions.

Cleanup is important—the Loop should delete worktrees upon task completion or handover. The loop-worktree CLI mechanizes this: one worktree per attempt, tracked in a manifest, cleaned up on rejection or escalation.

2.3 Skills — Persistent Memory of Intent

A Skill (typically SKILL.md + optional scripts/reference files) encodes:

Without Skills, the Loop re-derives everything from scratch on every run—this is Intent Debt. Skills are the way to pay down intent debt: conventions written once, read on every run.

2.4 Plugins & Connectors (MCP) — Connecting to the Real World

A Loop that can only read the filesystem is limited. Connectors enable the Loop to:

MCP (Model Context Protocol) has become the universal substrate; a connector written for one tool often works in another.

2.5 Sub-agents — Maker / Checker Separation

This is the most important structural pattern for a reliable Loop.

The Agent that writes code is a poor judge of its own work. A second Agent (sometimes using a stronger model, always using different instructions) performs verification.

Common separation patterns:

In an unattended Loop, the Verifier is the key that allows you (the human) to confidently walk away.

2.6 Memory / State — The Spine Across Sessions

Models have no long-term memory between independent turns or sessions. The Loop must read and write something persistent:

Good state answers three questions:

  1. What are we currently working on?
  2. What did we try last time, and what was the result?
  3. What is waiting for a human?

The state file is often the most important artifact a Loop produces.

2.7 Component Assembly Order

A minimum viable Loop typically starts with:

Scheduling + one Skill (triage) + State file

Then gradually adds:

The best Loops are those where each new component is added only after the previous version has proven its value (and failure modes).

3. A LangChain Perspective: The Four-Layer Loop Model

If Addy Osmani's six building blocks answer "what is a Loop composed of," then the four-layer Loop model proposed by the LangChain team in "The Art of Loop Engineering" answers "how Loops stack and continuously evolve." Swyx calls this art of layering Loopcraft—"the art of stacking loops."

These four layers are not parallel relationships but a bottom-up, layer-by-layer enhancement stacking structure. Each layer builds upon the one below it, and outer Loops can "reach in" to modify the configuration of inner Loops.

Image

3.1 Loop 1: Agent Loop — The Most Basic "Model Calls Tools"

This is the starting point for all Agents: give the LLM context, let it call tools, loop until the task is complete.

from langchain.agents import create_agent

# The most basic Agent Loop: Model + Tools = An Agent that can act
agent = create_agent(
    model="claude-sonnet-4-20250514",
    tools=[read_file, write_docs, open_pr],
    system_prompt="You are a documentation assistant, responsible for improving project documentation."
)

# One call, the Agent internally loops calling tools until completion
result = agent.invoke({
    "messages": [{"role": "user", "content": "Update the installation instructions in the README"}]
})

This is what LangChain's create_agent provides. Pick a model, plug in tools, and you have a working Agent Loop. Take LangChain's internal documentation Agent as an example: it receives documentation improvement requests → the model plans and drafts changes → uses tools to clone the repo, read/write files, and open a PR.

But the problem is obvious: The Agent's output is not necessarily correct or consistent. It might make mistakes on the first try, and you have no mechanism to discover them.

3.2 Loop 2: Verification Loop — "Writing isn't done until it's verified"

When consistency matters, wrap a verification loop around the Agent Loop: check the output, retry with feedback if it fails.

from langchain.agents import create_agent
from deepagents.middleware import RubricMiddleware

# RubricMiddleware implements the verification loop:
# Agent output → Grader checks against rubric → Retry with feedback if it fails
agent = create_agent(
    model="claude-sonnet-4-20250514",
    tools=[read_file, write_docs, open_pr],
    middleware=[
        RubricMiddleware(
            rubric="""
            1. All links must be resolvable
            2. All CI checks must pass
            3. The diff scope must not exceed the requested changes
            """,
            max_retries=3  # Max 3 retries
        )
    ]
)

The core of the verification loop is the Grader—it checks the Agent's output against a rubric. The Grader can be deterministic (like running tests, checking links) or Agentic (LLM-as-a-Judge, using another model to judge).

For the documentation Agent: the Grader runs tests after each attempt, checking if all links are resolvable, CI passes, and the diff is within the requested scope. These error types can be caught automatically without human review.

The cost: The verification loop adds latency and cost per run. But when quality matters more than speed—which is true in most production scenarios—this cost is worthwhile.

3.3 Loop 3: Event-Driven Loop — "The Agent is no longer something you manually invoke"

The first two layers automate "doing" and "verifying," but the Agent still requires you to trigger it manually. An event-driven loop connects the Agent to your ecosystem—a new document arrives, a timer fires, a webhook comes in—and the Agent runs automatically.

from langsmith import deployment

# LangSmith Deployment supports cron and webhook triggers
# Deploy the Agent as a continuously running background service
deployment.serve(
    agent=agent,
    triggers=[
        # Cron trigger: Run every day at 9 AM
        deployment.CronTrigger(schedule="0 9 * * *"),
        # Webhook trigger: Run when a Slack message arrives
        deployment.WebhookTrigger(
            source="slack",
            channel="#docs-plz"
        )
    ]
)

LangChain's documentation Agent achieves event-driven behavior through Fleet (a no-code Agent builder) channels and schedules: whenever a message appears in the #docs-plz Slack channel, it triggers the documentation Agent to run.

The key shift: The Agent is no longer something you manually invoke—it is a component continuously running within a larger system. This is also the essence of the "heartbeats" concept in OpenClaw: turning your Agent into an always-on, proactive assistant.

3.4 Loop 4: Hill-Climbing Loop — "Automating the improvement itself"

The first three layers automate the work. The fourth—and most important—layer automates the improvement.

from langsmith import Engine

# LangSmith Engine analyzes production Traces, automatically improves Harness configuration
engine = Engine(
    agent=agent,
    analysis_prompt="""
    Analyze recent Agent run Traces to find:
    1. Which prompts led to low-quality output?
    2. Which tool calls frequently fail?
    3. Does the Grader's rubric need adjustment?
    """,
    auto_improve=True  # Automatically apply improvements
)

# Engine continuously monitors, automatically submits improvement PRs when issues are found
engine.monitor()

Every Agent run produces a Trace: what the model did, which tools it called, what feedback the Grader gave. These Traces contain high-value signals about "what works and what doesn't." The hill-climbing loop runs an analysis Agent to process these Traces, using the findings to rewrite the Harness configuration—including prompt adjustments, tool configuration optimization, and Grader standard improvements.

The key action: The return arrow doesn't just go back to the top layer—it reaches in and directly updates the Agent Loop. Each cycle of the outer Loop makes the inner Loop more effective.

For teams going further, the hill-climbing loop can also:

3.5 Overview of the Four-Layer Loop

Loop What it does Impact LangChain Primitive
1. Agent Loop Model repeatedly calls tools until task completion Automates work create_agent
2. Verification Loop Agent output checked against rubric, retried with feedback if failed Ensures quality and correctness RubricMiddleware
3. Event-Driven Loop Events trigger Agent runs, updating real systems Automates work at scale LangSmith Deployment (cron/webhook) or Fleet channels
4. Hill-Climbing Loop Production Traces fed to analysis Agent, improving Harness config Continuous Harness improvement LangSmith Engine

The LangChain team's core argument: We have thought about Loop 1 and Loop 2 for a long time. But the focus should shift to Loop 3 and Loop 4—embedding Agents into your ecosystem and letting them continuously improve against your standards. That's where the compounding value lies.

Satya Nadella has a pithy summary: "Those who build learning loops early—where human judgment and token capital compound together—will build an advantage that is hard to replicate."

3.6 Relationship Between the Four-Layer Model and the Six Building Blocks

LangChain's four-layer model and Addy Osmani's six building blocks are complementary perspectives, not competing ones:

Six Building Blocks Position in the Four-Layer Model
Scheduling Core of Loop 3 — cron/webhook triggers
Worktrees Isolation foundation for Loop 1/2 — parallel safety
Skills Context injection for Loop 1 + Optimization target for Loop 4
MCP Connectors Tool layer for Loop 1 + Event sources for Loop 3
Sub-agents Maker/Checker separation for Loop 2
State/Memory Persistent spine across all Loops

Simply put: The six building blocks are the "parts list," and the four-layer model is the "assembly manual."

4. How to Design a Loop

4.1 Anatomy of a Loop

A complete Loop run cycle includes the following phases:

Image

4.2 Autonomy Levels: L0 → L1 → L2 → L3

Loop Engineering emphasizes phased delegation of authority, never in one giant leap. The full autonomy levels start from an L0 draft:

Level Description
L0 Draft Loop design document + manual trigger, verify process correctness
L1 Report Only Triage → Write to state, no automatic actions taken
L2 Assisted Small automatic fixes + Verifier validation
L3 Unattended Autonomous operation without human monitoring

L0 Draft is the starting point most teams ignore: You first write a LOOP.md design document, manually run each step, and confirm that the triage logic, state read/write, and escalation triggers all meet expectations. The L0 phase involves no automation—you are verifying "is the design of this Loop itself correct."

Key principle: Never skip L1 on a new pattern's production repository. From L0 to L3, each level must only be advanced after the previous level has run stably.

Image

4.3 Prerequisites for Loop Design

Before enabling a production Loop, you must pass checks across the following 10 dimensions:

1. Purpose and Scope

2. Scheduling

3. Skills

4. Maker / Checker Separation

5. State / Memory

6. Human Handover

7. Connectors (MCP)

8. Cost and Limits

9. Observability

10. Security

4.4 Guardrails for Loop Design

If a Loop breaks, stop and fix before continuing. Reference examples:

4.5 Anti-Patterns of Loop Design

The following anti-patterns are mistakes that must be avoided at the design stage—they are not runtime failures, but fundamental architectural flaws. Before enabling an unattended Loop, check against each one:

# Anti-Pattern Why It's Fatal Correct Approach
1 Same Agent implements and verifies Checking code you wrote yourself is no check at all. Models unconsciously favor their own implementation. Implementer and Verifier must be different Agents, different models, or different instructions
2 No Early Exit Full scan even when the monitoring list is empty, burning millions of tokens daily for nothing Exit within ~3-5k tokens on an empty list, don't run an extra lap "just in case"
3 State file is a sham State is written but not read on the next run, or the state schema is unclear, effectively starting from zero each time State file must have a documented schema, previous state must be forcibly read at the start of each run
4 Auto-merge without a branch allowlist Loop can merge directly to main, polluting the trunk immediately on error Auto-merge must have dual allowlists for branches and paths
5 "Fixing" flake tests with retries Intermittent test failures are masked by retries, real bugs are buried Flake tests should be marked @flake and excluded, or escalated to a human
6 Notifying on every run The team quickly learns to ignore Loop messages, missing the ones that truly need attention Ping only when human action is needed—"Found 3 high-priority Issues" is more valuable than "Run complete"
7 No token budget cap A runaway Loop can burn 8 million tokens in 48 hours Set a daily token cap, switch to report-only mode at 80%, terminate at 100%
8 Verifier shares a session with Implementer Context pollution—Verifier sees the Implementer's reasoning process, losing independence Verifier must run in a fresh session/worktree, seeing only the code diff
9 Infinite fix loop The same Issue repeatedly fails to fix, the Loop never stops Max 3 attempts per item, auto-escalate to human after exceeding
10 Skipping L1 straight to L3 Jumping from "report only" directly to "auto-merge" with no verification in between Must go through L0→L1→L2→L3, each level stable for 1-2 weeks before advancing

The most important anti-patterns are #1 and #8—they essentially say the same thing: Independence is the prerequisite for verification. A verifier that has seen the implementation process, no matter how strong, has already lost the ability to judge independently.

4.6 Incident Response Process for Loop Design

When a Loop has an incident in production (e.g., auto-merged wrong code, burned through the budget, modified files it shouldn't have), handle it with the following four-step process:

Image

Step 1: Stop Immediately

Execute loop-pause-all (globally pause all Loops) or scheduler_delete (delete the scheduling task for a specific Loop). This is not a "suggested pause"—it is immediately, right now, stop.

Step 2: Root Cause Analysis

Don't rush to fix. First ask three questions:

  1. Which safety mechanism should have prevented this but didn't?
  2. At what level was this Loop running? Were levels skipped?
  3. What do the state file and run logs say?

Step 3: Fix and Downgrade

After fixing the root cause, do not restore the original level. Downgrade the Loop by at least one level:

Step 4: Document the Lesson

Write an honest post-mortem in the stories/ directory, format reference why-we-killed-ci-sweeper.md. The value of this document is not "what we learned"—but that the next team doesn't need to step on the same rake.

5. Loop Pattern Design

Image

5.1 Daily Triage

Goal: Every morning (or active period), get a prioritized, actionable picture of the to-do list—without manually checking CI, issues, PRs, and chat.

Typical Cycle: Schedule triggers → Triage Skill ingests CI failures (24h), open issues/tickets, recent commits, previous STATE.md → High-priority items appended to state with suggested next actions → Clean up resolved items → Record post-run review.

The best starter Loop. Run report-only mode for 1-2 weeks first, measure triage accuracy before enabling L2.

5.2 PR Babysitter

Goal: Reduce human time spent on PR review, CI, rebase, and merge, while keeping humans in the judgment seat.

Typical Cycle: Discover team's open PRs → Run triage on each PR → Generate fix if CI is red → Propose minimal patch if there are review comments → Mark "ready to merge" if ready → Escalate to human if ambiguous or high-risk.

5.3 CI Sweeper

Goal: Rapidly respond to CI failures on main or active branches—diagnose, propose minimal fixes, escalate when unable to resolve confidently.

Key Safety Measures: Classify flake vs real regression vs infrastructure; no automatic fix for flakes; max 3 attempts for the same failure before escalation; loop-guard circuit breaker prevents infinite loops.

5.3.1 Real Failure Case: Why We Killed Our CI Sweeper

loop-engineering documents a real incident—this is exactly why CI Sweeper is marked as "L2 with caution":

Dimension Details
Trigger loop 5m running while the main branch was red, coinciding with a flaky migration branch
Damage Burned ~8 million tokens in 48h; 3 out of 11 PRs were symptomatic fixes, 1 broke production config (caught by a human)
Root Cause 1 No L1 phase—jumped straight to auto-fix
Root Cause 2 Verifier and Implementer were in the same session (half the runs)
Root Cause 3 No branch allowlist—swept feature branches with known red CI
Root Cause 4 No daily budget—scheduler never paused
Post-Mortem Deleted all schedule tasks → Switched to event-driven on main failure → Report-only for 1 week → Verifier as independent sub-Agent → Daily 2M token cap → Branch allowlist only main

Lesson distilled: This case exposes the most dangerous cognitive bias in Loop Engineering—"automation = safety." In reality, the higher the degree of automation, the stricter the safety mechanisms required. The CI Sweeper pattern documentation exists precisely because someone paid a real monetary price.

Remember: L2 is not an upgraded version of L1, but a cautious extension of L1 after adding a safety net.

5.4 Dependency Sweeper

Goal: Automate patch and low-risk CVE dependency updates, reducing "dependency staleness anxiety."

Typical Cycle: Schedule triggers (recommended weekly, off-peak hours) → Scan package.json / requirements.txt / Cargo.toml → Filter: patch versions only + low-risk CVEs → Create independent Worktree for each candidate dependency → Run npm update / pip install --upgrade → Execute test suite → Create PR if tests pass, log reason and skip if they fail.

Key Safety Measures:

Why start with patches: Patch version updates are the most "boring" work—clear benefits (bug fixes, vulnerability patches), extremely low risk (SemVer guarantees backward compatibility). Let the Loop handle the boring stuff, humans handle the stuff requiring judgment.

5.5 Changelog Drafter

Goal: Automatically generate release note drafts from merged PRs and commits, eliminating the pain of "rushing to write the changelog before release."

Typical Cycle: Schedule triggers (recommended before each release, or every Friday afternoon) → Pull all merged PRs since the last release → Categorize by label (bug, feature, breaking, docs, deps) → Extract key info from PR titles and descriptions → Generate a Markdown draft by category → Annotate sections needing human supplementation (e.g., "The migration guide for this breaking change needs you to write it").

Key Design Decisions:

Low risk, high value. This is the most suitable pattern for a second Loop—after Daily Triage is stable, Changelog Drafter has almost zero risk but significantly reduces release friction.

5.6 Post-Merge Cleanup

Goal: Identify content that can be cleaned up after a merge—dead code, outdated comments, TODO debt, unused imports.

Typical Cycle: Schedule triggers (recommended off-peak hours, like 2 AM) → Scan files involved in recently merged PRs → Detect patterns: commented-out code blocks, TODO / FIXME markers, unused imports, duplicate code snippets → Generate cleanup suggestions for each finding → Create a single "cleanup PR" (rather than one PR per finding).

Key Safety Measures:

Why this pattern is needed: Codebase entropy is inevitable. Every merge adds complexity—temporary code becomes permanent, comments become outdated, TODOs are never done. Post-Merge Cleanup is an automated means to fight entropy.

5.7 Issue Triage

Goal: Automatically classify new Issues—mark duplicates, suggest priority, route to the correct team. Suggestions only, no automatic closing.

Typical Cycle: Webhook triggers (on new Issue creation) → Read Issue title and body → Similarity match against existing Issues (detect duplicates) → Check information completeness against a template (are there reproduction steps, environment info, expected behavior) → Suggest labels (bug / feature / question / duplicate) and priority → If information is insufficient, generate a friendly follow-up comment → Write triage results to STATE.md.

Key Design Decisions:

Relationship with Daily Triage: Issue Triage is event-driven "real-time triage," Daily Triage is schedule-driven "global triage." The two complement each other—Issue Triage ensures new Issues aren't missed, Daily Triage ensures the global perspective isn't lost.

5.8 Pattern Selection and LangChain Mapping

The seven patterns above cover the most common automation scenarios in software engineering. Which pattern to choose depends on your pain points and risk tolerance—not "which pattern is cooler."

Selection Guide:

Your Pain Point Recommended Pattern Starting Level Why Start at This Level
"Don't know the project status every morning" Daily Triage L1 Zero risk, pure information output
"PR review takes too much time" PR Babysitter L1 → L2 First see the quality of Agent's suggestions, then enable auto-fix
"CI is often red, no one checks promptly" CI Sweeper L1 → L2 Must start from L1—refer to 5.3.1 incident case
"Dependency updates are always forgotten" Dependency Sweeper L2 patches only Patch updates have very low risk, suitable for direct L2
"Writing release notes is painful" Changelog Drafter L1 Zero risk, pure text generation
"Codebase is getting messier" Post-Merge Cleanup L1 First see if cleanup suggestions are reasonable
"Issue backlog is severe" Issue Triage L1 Suggest only, no execution, zero risk

Mapping Patterns to the LangChain Four-Layer Model:

Each pattern can be implemented with LangChain primitives, mapped as follows:

Pattern Core Loop Layers LangChain Key Primitives Description
Daily Triage Loop 1 + Loop 3 create_agent + CronTrigger Agent performs triage, cron triggers on schedule
PR Babysitter Loop 1 + Loop 2 create_agent + RubricMiddleware Agent fixes + Grader verifies
CI Sweeper Loop 1 + Loop 2 + Loop 3 create_agent + RubricMiddleware + WebhookTrigger CI event triggers, verify after fix
Dependency Sweeper Loop 1 + Loop 3 create_agent + CronTrigger Scheduled check + auto patch PR
Changelog Drafter Loop 1 + Loop 3 create_agent + CronTrigger Scheduled generation of release note draft
Post-Merge Cleanup Loop 1 + Loop 3 create_agent + CronTrigger Off-peak scheduled cleanup
Issue Triage Loop 1 + Loop 3 create_agent + WebhookTrigger Issue event triggers triage

Advanced Practice: When all patterns are running stably, introduce Loop 4 (LangSmith Engine) to perform unified analysis on the production Traces of all patterns. The Engine can discover cross-pattern improvement opportunities—for example, when CI Sweeper and PR Babysitter are repeatedly fixing the same type of problem, the Engine can suggest merging or adjusting the division of labor. This is the true value of the "hill-climbing loop": not improving a single Loop, but optimizing the entire Loop ecosystem.

5.9 Design Principles Behind the Patterns

The seven patterns seem different, but they share five design principles behind them. Understand these principles, and you can design your own patterns, not just copy others'.

Principle 1: Start from L1, always start from L1. This is not conservatism, it's engineering discipline. L1 lets you verify the Agent's judgment quality at zero risk. Run Daily Triage for two weeks report-only, and you'll know its triage accuracy. The CI Sweeper incident happened precisely because this step was skipped. L1 is not "incomplete functionality," but a "necessary observation period before the safety net is in place."

Principle 2: Maker/Checker separation is not optional. Among the seven patterns, any involving code modification (PR Babysitter, CI Sweeper, Dependency Sweeper, Post-Merge Cleanup) requires the Implementer and Verifier to be independent sub-Agents. This is not over-engineering—in the CI Sweeper incident, half the runs had the Verifier and Implementer in the same session, creating a blind spot of "verifying one's own work." AI cannot grade its own code.

Principle 3: The smaller the scope, the higher the trust. Dependency Sweeper only does patch updates (very small scope) → can go directly to L2. CI Sweeper needs to diagnose arbitrary CI failures (very large scope) → must start from L1. This is not a double standard, but matching risk to scope. When designing a pattern, first ask "how big is this Loop's blast radius," then decide the autonomy level.

Principle 4: Event-driven is better than scheduled polling—but with a prerequisite. Issue Triage uses Webhook triggers (event-driven), which is more timely and token-efficient than scheduled scanning. But the CI Sweeper incident happened precisely because scheduled polling (loop 5m) repeatedly triggered at the wrong time. The prerequisite for event-driven is: you clearly know "what events are worth responding to." If unsure, use scheduled + L1 observation first.

Principle 5: Every pattern must answer "when to escalate to a human." This is not a fallback for "what if something goes wrong," but a core design constraint of the pattern. PR Babysitter's escalation condition is "ambiguous or high-risk situation," CI Sweeper's is "3 failed attempts," Dependency Sweeper's is "major version or blacklisted package." A Loop without clearly defined escalation conditions is not a complete Loop.

6. Example Code (LangChain)

Below, we use the LangChain ecosystem to implement the core patterns of Loop Engineering. LangChain provides complete primitives from basic Agent Loops to event-driven, verification loops, and hill-climbing loops, allowing us to build Loops directly at the framework level without writing schedulers and state management from scratch.

6.1 Project Structure

my-loop-project/
├── agent.py                # Agent definition and configuration
├── tools/
│   ├── github_tools.py     # GitHub PR/Issue tools
│   └── ci_tools.py         # CI status query tools
├── skills/
│   ├── triage.md           # Triage Skill (Markdown format)
│   └── minimal_fix.md      # Minimal fix Skill
├── middleware/
│   └── rubric.py           # Verification rubric
├── deployment.py           # Event-driven deployment configuration
└── engine_config.py        # Hill-climbing loop (Engine) configuration

6.2 Loop 1: Agent Loop — create_agent

The most basic Agent Loop: Model + Tools + System Prompt.

# agent.py
from langchain.agents import create_agent
from langchain.chat_models import init_chat_model

# Initialize model
model = init_chat_model("claude-sonnet-4-20250514")

# Define tools
from tools.github_tools import (
    list_open_prs, get_pr_details, create_pr, add_pr_comment,
    get_ci_status, get_recent_commits
)

# Create Agent — this is Loop 1
agent = create_agent(
    model=model,
    tools=[
        list_open_prs,      # List open PRs
        get_pr_details,     # Get PR details
        get_ci_status,      # Query CI status
        get_recent_commits, # Get recent commits
        create_pr,          # Create PR
        add_pr_comment,     # Add PR comment
    ],
    system_prompt="""You are a PR Babysitter Agent.
Check the status of all open PRs every morning:
1. If CI fails, analyze the cause and suggest a fix
2. If there are unanswered review comments, generate a minimal patch
3. If everything is ready, mark as ready-to-merge
4. If the situation is ambiguous or high-risk, escalate to a human
"""
)

# Single invocation
result = agent.invoke({
    "messages": [{"role": "user", "content": "Check the status of all open PRs"}]
})

6.3 Loop 2: Verification Loop — RubricMiddleware

Wrap a verification layer around the Agent Loop: Output → Grade → Retry with feedback if failed.

# middleware/rubric.py
from deepagents.middleware import RubricMiddleware

# Define the rubric
pr_review_rubric = RubricMiddleware(
    rubric="""
    For each PR fix output by the Agent, score against the following criteria (0-10 per item):

    1. Correctness (Weight 40%): Does the fix truly resolve the CI failure or review comment?
       - 10 points: Fix precisely targets the issue, introduces no new errors
       - 0 points: Fix is unrelated to the issue, or introduces new problems

    2. Minimality (Weight 30%): Is the diff as small as possible?
       - 10 points: Only changed necessary lines, no unrelated changes
       - 0 points: Refactored unrelated modules, or changed more than 10 files

    3. Safety (Weight 30%): Did it touch blacklisted paths?
       - 10 points: Did not touch sensitive paths like auth/, payments/, secrets/
       - 0 points: Touched blacklisted paths → Escalate immediately

    Total score below 7 → Retry with feedback
    Touched blacklist → Do not retry, escalate directly to human
    """,
    max_retries=3,  # Max 3 retries
    on_max_retries="escalate"  # Escalate after exceeding retries
)

# Mount the verification middleware onto the Agent
agent_with_verification = create_agent(
    model=model,
    tools=[list_open_prs, get_pr_details, get_ci_status,
           create_pr, add_pr_comment, get_recent_commits],
    middleware=[pr_review_rubric],
    system_prompt="You are a PR Babysitter Agent..."
)

6.4 Loop 3: Event-Driven Loop — LangSmith Deployment

Deploy the Agent as a continuously running background service, triggered via cron or webhook.

# deployment.py
from langsmith import deployment

# Deploy Agent, configure triggers
deployment.serve(
    agent=agent_with_verification,
    name="pr-babysitter",
    triggers=[
        # Cron trigger: Check PR status every 15 minutes
        deployment.CronTrigger(
            schedule="*/15 * * * *",
            input_template="Check the status of all open PRs"
        ),
        # Webhook trigger: Respond immediately on new PR or CI status change
        deployment.WebhookTrigger(
            source="github",
            events=["pull_request.opened", "check_run.completed"]
        ),
    ],
    # Runtime configuration
    runtime={
        "max_concurrency": 3,       # Max 3 concurrent runs
        "timeout_seconds": 600,     # Single run timeout 10 minutes
        "error_policy": "escalate", # Escalate to human on error
    }
)

6.5 Loop 4: Hill-Climbing Loop — LangSmith Engine

Analyze production Traces, automatically improve Harness configuration.

# engine_config.py
from langsmith import Engine

# Engine continuously analyzes Agent run Traces, automatically improves
engine = Engine(
    agent=agent_with_verification,
    analysis_prompt="""
    Analyze the Traces of the last 100 Agent runs to find improvement points:

    1. Prompt Issues:
       - Which system prompts led to low-quality output?
       - Does the Agent frequently misunderstand certain instructions?

    2. Tool Issues:
       - Which tool calls frequently fail or timeout?
       - Are tool descriptions accurate?

    3. Grader Issues:
       - Is the rubric too lenient/strict?
       - Are there cases of "passed verification but actually problematic"?

    4. Pattern Issues:
       - Which types of PR fixes have the lowest success rate?
       - Are there recurring failure patterns?

    For each issue found, propose specific improvement suggestions.
    """,
    auto_improve=True,  # Automatically apply non-destructive improvements
    improvement_policy={
        "prompt_tweaks": "auto",       # Prompt tweaks auto-applied
        "tool_config": "auto",         # Tool config auto-applied
        "rubric_changes": "review",    # Rubric changes require human review
        "model_switch": "review",      # Model switch requires human review
    },
    schedule="0 2 * * 1",  # Run analysis every Monday at 2 AM
)

# Start Engine monitoring
engine.monitor()

6.6 Human-in-the-Loop: Human Intervention Points

LangChain treats Human-in-the-Loop as a first-class citizen, allowing human review to be inserted at every Loop layer:

from langchain.agents import create_agent
from deepagents.middleware import HumanInTheLoopMiddleware

agent_with_human = create_agent(
    model=model,
    tools=[create_pr, add_pr_comment, merge_pr],
    middleware=[
        # Require human confirmation before sensitive operations
        HumanInTheLoopMiddleware(
            require_approval_for=[
                "merge_pr",           # Merging PR requires human confirmation
                "create_pr",         # Creating PR requires human confirmation (optional)
            ],
            auto_approve_for=[
                "add_pr_comment",    # Adding comments auto-approved
            ],
            approval_timeout_hours=24,  # Auto-reject if no response within 24 hours
        ),
        pr_review_rubric,
    ],
    system_prompt="You are a PR Babysitter Agent..."
)

6.7 Complete Example: Daily Triage Loop

Assemble the above components into a complete Daily Triage Loop:

# daily_triage.py — Complete Daily Triage Loop
from langchain.agents import create_agent
from langchain.chat_models import init_chat_model
from deepagents.middleware import RubricMiddleware
from langsmith import deployment

model = init_chat_model("claude-sonnet-4-20250514")

# Triage Agent: Read-only, report-only, no modifications
triage_agent = create_agent(
    model=model,
    tools=[
        get_ci_status,
        list_open_prs,
        get_recent_commits,
        list_open_issues,
    ],
    system_prompt="""You are the Daily Triage Agent.
Scan project status every morning:
1. Check CI status (failures in the last 24 hours)
2. List all open PRs and their status
3. Check for outdated dependencies
4. Write findings to STATE.md

Rules:
- Report only, do not automatically perform any actions
- Mark high-priority items as needing human attention
- Clean up resolved items from STATE.md
""",
    middleware=[
        RubricMiddleware(
            rubric="""
            Triage results must:
            1. Have a clear priority (high/medium/low) for each finding
            2. Have a suggested next action for each finding
            3. Not miss any CI failures or PRs inactive for over 3 days
            """,
            max_retries=2
        )
    ]
)

# Deploy to run every weekday at 9 AM
deployment.serve(
    agent=triage_agent,
    name="daily-triage",
    triggers=[
        deployment.CronTrigger(
            schedule="0 9 * * 1-5",  # Weekdays at 9 AM
            input_template="Execute daily triage, scan CI, PRs, Issues, and dependency status"
        )
    ]
)

7. Advanced Loop Engineering

7.1 Cost Management

Token cost is the most realistic constraint of Loop Engineering. A poorly designed Loop can burn millions of tokens in a single day.

Cost Estimation Formula

Daily Tokens ≈ Tokens per Run × Runs per Day × (1 + Sub-Agent Multiplier)
Factor Impact
Cadence Linear multiplier (5 min vs 1 day = 288× runs/day)
Sub-Agents per Run Each = Full model + tool roundtrips
Context Size Large repo + full CI logs = Expensive triage
Verifier Model Using a stronger model on Verifier = Worth it (when unattended)

Cost Control Best Practices

Triage First: Use a cheap triage scan first (~5k tokens), only launch sub-Agents when actionable items are found. Early Exit: Exit within ~3-5k tokens when the monitoring list is empty. Budget Cap: loop-budget.md sets a daily token cap, auto-pause when exceeded. Circuit Breaker: Auto-escalate the same item after 3 attempts, preventing infinite fix loops. Off-Peak Slowdown: Use slower cadences at night and on weekends.

Slow Down / Pause / Kill: Three-Tier Emergency Standards

Cost control isn't just about "saving money"—when a Loop behaves abnormally, you need clear emergency standards. Here are the three-tier responses for a production-grade Loop:

Tier Trigger Condition Action Taken Recovery Method
🟡 Slow Down Single run token exceeds 50% of budget, or 2 consecutive runs with no progress Cadence halved, switch to a cheaper model for triage Automatically speed up when the next run returns to normal
🟠 Pause Daily tokens reach 80% of budget, or loop-pause-all activated Exit after current round completes, scheduler marked as paused Manually execute loop-resume to recover
🔴 Kill Daily tokens reach 100% of budget, or dangerous path modification detected Immediately terminate the current run, do not wait for completion Manually restart from L0 after root cause analysis

loop-pause-all is the global emergency switch—after execution, all Loops exit immediately after the current round. It's not a "suggested pause," but a hard termination. Coupled with the rule If loop-pause-all is active, exit immediately in loop-constraints.md, it ensures that even if the scheduler can't cancel in time, the Loop itself will actively exit.

7.2 State Synchronization

When multiple Loops or humans modify state simultaneously, drift occurs. Codex CLI's state subcommand can detect inconsistencies between STATE.md and LOOP.md:

# Check state consistency
codex state diff --state STATE.md --loop LOOP.md

# Auto-resolve safely mergeable conflicts
codex state sync --state STATE.md --loop LOOP.md --auto-resolve

State Design Principles:

7.3 Context Management (loop-context)

Codex CLI has built-in stateful memory management and long-run circuit breakers, implemented via the context subcommand:

# Inject historical state into the current run context
codex context inject --state STATE.md --since "24h"

# Prune outdated entries, preventing infinite state file growth
codex context prune --state STATE.md --older-than "7d"

# Detect repeated attempts on the same failure, trigger escalation
codex context guard --state STATE.md --max-retries 3

It solves three problems:

1. Context Injection: Inject relevant historical state at the start of each run. 2. Context Pruning: Clean up outdated entries, preventing infinite state file growth. 3. Circuit Breaker: Detect repeated attempts on the same failure, trigger escalation.

7.4 Worktree Management (loop-worktree)

Codex CLI manages isolated git worktrees for each fix attempt via the worktree subcommand:

# Create an isolated worktree
codex worktree create --run-id <id> --pattern <p>

# Lock paths (prevent multi-Loop conflicts)
codex worktree lock --paths <globs> --owner <pattern>

# Unlock
codex worktree unlock --owner <pattern>

# Clean up completed or abandoned worktrees
codex worktree cleanup --older-than "1h"

Key Principle: One worktree per attempt, tracked in a manifest, cleaned up on rejection or escalation.

7.5 Goal Engineering

Goal Engineering is a companion concept to Loop Engineering. If Loops are responsible for "discovery and persistence," Goals are responsible for "focus and completion":

Loops discover, goals finish.

A Goal is a verifiable stop condition—the Agent iterates continuously until the condition is met:

/goal All tests on main pass and lint is clean
/goal Keep working on this PR until CI is green and no blocking comments remain

Goals use a fresh model to judge whether the stop condition is met—another application of the Maker/Checker idea.

7.6 Loop-Gate

Codex CLI's gate subcommand mechanically enforces path blacklists and auto-merge allowlists from gate.yaml:

# Check if auto-merge is allowed
codex gate check --action auto-merge --paths <f1,f2,...>

# Check if paths are on the blacklist
codex gate check --action deny --paths <f1,f2,...>

Exit code convention:

This uses the same convention as codex context guard, so control scripts can chain both.

Loop Constraints

loop-constraints.md is a higher-level safety mechanism than gate.yaml—it is a set of hard rules written in natural language that the Loop reads verbatim and obeys at the start of each run. Complementing the mechanical path checks of gate.yaml, loop-constraints covers constraints at the "intent level."

A typical loop-constraints.md contains rules across five dimensions:

Dimension Example Rules Why Needed
Push & Merge Never auto-merge to main; create draft PRs first Prevent unreviewed code from entering trunk
Paths Never edit .env, auth/, payments/, secrets/ Protect sensitive configs and credentials
Code Run tests before every fix; never disable tests to make CI green; max 3 attempts per run Prevent fake fixes and infinite loops
Communication Tell me before you do it; don't close Issues/PRs without approval Keep humans in control of key decisions
Budget Switch to report-only when tokens reach 80% daily cap; exit immediately when loop-pause-all is active Cost safety net

Constraints are binding—the Agent must obey them. They are not "flexibly interpreted" like system prompts, but are written into the Loop's run flow as hard checks. Coupled with Codex CLI's context guard mechanical enforcement (recording each attempt to loop-ledger.json, running codex context guard --check before retries), constraints transform from "suggestions" into "unbypassable guardrails."

7.7 Multi-Loop Coordination

Running multiple Loops in one repository is normal. Running them without boundaries is how Loops fight each other.

Core Principles

1. One branch, one owner — At most one Loop can modify a branch per hour.

2. Separate state files — STATE.md for triage; pattern-specific files for operational Loops.

3. Triage reports, operational Loops execute — L1 Daily Triage never competes with CI Sweeper fixes.

4. Shared blacklist — Copy the same path blacklist into every LOOP.md.

5. Aggregate token budget — All Loops share one budget cap.

Conflict Priority

Image

7.8 Three "Debts" That Must Be Confronted

Intent Debt

Every session, the Agent is a "cold start." Team conventions, build commands, "we never do that"—if not written into Skills / AGENTS.md, every loop round is re-guessing. Skills are the way to pay down intent debt.

Comprehension Debt

The faster the Loop, the more code in the repository that "you wrote but haven't read." The Loop delivered it, but that doesn't mean you understand it. Faster Loops ship more code you didn't write—comprehension debt grows unless you read what the Loop did.

Cognitive Surrender

The most dangerous usage: treating the Loop as a button to avoid thinking. Addy Osmani warns:

Build the loop. But build it like someone who intends to stay the engineer, not just the person who presses go.

The same Loop design can accelerate a real engineer, or accelerate a "person who just presses Go"—the difference is whether you encoded your judgment into the Skills and Verifier.

7.9 Top Ten Common Failure Modes

1. Infinite Fix Loop: The same PR is auto-fixed 5+ times, never converging. Mitigation: Hard cap of 3 times → escalate to human.

2. State Rot: STATE.md references merged PRs, closed tickets. Mitigation: Clean up on every run.

3. Verifier Theater: Verifier "approves" but CI tests fail. Mitigation: Verifier must run tests and report output.

4. Notification Fatigue: Pings every 5 minutes; team mutes the bot. Mitigation: Notify only when human decision is needed.

5. Token Burn: Bill skyrockets; Loop runs full sub-Agent chains on empty or noisy triage. Mitigation: Cheap triage first.

6. Scope Creep: Loop refactors unrelated modules. Mitigation: Path blacklist + "smallest possible diff."

7. Comprehension Debt Spiral: Speed goes up, but no one can explain recent changes. Mitigation: Mandatory human review for non-trivial PRs.

8. Cognitive Surrender: "The Loop will handle it"—no opinion on correctness or design. Mitigation: Explicit human gates in every pattern.

9. Parallel Collision: Two sub-Agents edit the same file. Mitigation: Worktree isolation + state locks.

10. Escalation Failure: Loop stuck in retries; human never notified. Mitigation: Connectors ping on escalation.

8. Conclusion and Future Outlook

Image

Trend 1: From "Writing Code" to "Designing Systems"

The engineer's role is shifting from "code producer" to "system designer." The future senior engineer is not the one who writes the most, but the one who designs the best Loop systems. Boris Cherny's practice has already proven this—he manages 15+ parallel Agent instances simultaneously, focusing on code review and direction setting.

Trend 2: Loop as Code

Just as Infrastructure as Code changed operations, Loop as Code will change software development. Loop configurations (LOOP.md, gate.yaml, loop-constraints.md) will become standard components of a project, just like CI configurations.

Trend 3: Standardization of Multi-Agent Orchestration

MCP (Model Context Protocol) is becoming the universal substrate for inter-Agent communication. In the future, Agents from different vendors will be able to collaborate via standardized protocols—a Claude Code Agent doing triage, a Codex Agent doing fixes, a Gemini Agent doing verification.

Trend 4: Convergence of Goal Engineering and Loop Engineering

Loops handle "discovery and persistence," Goals handle "focus and completion." The convergence of the two will produce more powerful autonomous development systems—Loops discover what needs to be done, Goals ensure each thing is done right.

Trend 5: Security and Governance Become First-Class Citizens

As Loops move from L1 to L3, security mechanisms will shift from "best practices" to "hard requirements." Path blacklists, auto-merge allowlists, MCP permission minimization, human gates—these will become standard for any production Loop.

Trend 6: Automation of Cost Optimization

Future Loop systems will have built-in intelligent cost management—automatically adjusting cadence based on budget, using cheaper models on low-value tasks, using stronger models on critical verification. Token budgets will shift from static configuration to dynamic optimization.

Trend 7: From Personal Tool to Team Infrastructure

Loop Engineering is currently mainly a personal practice (Boris's 15 parallel Agents), but it is rapidly evolving into team infrastructure. Shared Skills libraries, team-level status boards, cross-project Loop patterns—these will become standard configurations for engineering organizations.

References

  1. https://addyosmani.com/blog/loop-engineering/
  2. https://www.langchain.com/blog/the-art-of-loop-engineering
  3. https://www.oreilly.com/radar/loop-engineering/
  4. https://github.com/alchaincyf/loop-engineering-orange-book
  5. https://github.com/cobusgreyling/loop-engineering