跪拜 Guibai
← All articles
Agent · Next.js

Making Multi-Agent Output Trustworthy with a Runtime Control Layer

By 倾颜 ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Prompt-based guardrails for agent systems are soft constraints that LLMs can ignore or be tricked into bypassing. This architecture shows how to build verifiable, hard guarantees into a multi-agent pipeline using strict schemas and pure-function state machines, a pattern directly applicable to any team moving agents toward production where output trustworthiness is non-negotiable.

Summary

AI Mind v0.4.11 tackles the core problem of multi-agent collaboration: LLM output is unstable, and chaining agents amplifies that uncertainty. The system generates delivery plans through a fixed chain of Supervisor, Plan, Task, and three Reviewer agents. Instead of trying to make the models more accurate, it introduces a deterministic control layer in the runtime that takes decision-making power away from the LLMs. Every agent output passes through a strict, role-specific schema validation; safety rules like mandatory reviewer counts and blocking verdicts are enforced by hard-coded logic, not prompts; and the feedback loop is capped at a single revision before handing final judgment back to a human. The design explicitly rejects ReAct loops, multi-round review cycles, and generic DAG scheduling as unnecessary complexity for this scope. A controlled evaluation against single-agent and unrevised multi-agent baselines showed significant gains in state consistency and rule reliability with manageable cost increases.

Takeaways
Every agent output is validated against a strict, role-specific Zod schema before any business logic consumes it; unknown fields cause a hard rejection, not silent deletion.
Structured encoding uses a separate, fixed model (deepseek-v4-pro) from the user-chosen business model, so switching models doesn't destabilize system judgments.
Safety rules like exact reviewer counts and blocking verdicts are enforced by hard-coded runtime functions, not by prompt instructions that agents can ignore.
A pure function resolves the final review bundle status using only structured coverage and findings data, with an immutable priority chain: execution failure > hard block > incomplete coverage > required fixes > pass.
Revision targets are derived directly by the runtime from validated findings; the Supervisor's guidance can only influence explanatory text, not the decision to revise or what to revise.
The system performs at most one revision round and never re-reviews its own output, fixing the final state to 'needs_review' so a human makes the final call.
ReAct loops, multi-round review cycles, generic DAG scheduling, persistence, and LLM report polishing are all deliberately excluded as out of scope for this version.
Conclusions

Treating LLM self-review as unreliable and capping the feedback loop at one revision is a pragmatic admission that automatic convergence is a fantasy at current capability levels.

Separating the model that does creative business judgment from the model that enforces output structure is an underused pattern that decouples user preference from system reliability.

The deliberate choice to reject unknown fields rather than silently strip them treats schema violations as boundary overreach, which is a security posture more than a parsing convenience.

Hard-coded gates sacrifice flexibility, but the argument here is that safety-critical rules should not be flexible; the trade-off is framed as correctness over adaptability.

Concepts & terms
Strict Schema Validation
A mode where a data structure definition rejects any input containing fields not explicitly declared, rather than silently dropping them. Used here to prevent agents from overstepping their role boundaries.
Pure Function State Machine
A function that always produces the same output for the same input with no side effects or randomness. The review status resolver uses this to guarantee deterministic decisions from structured data alone.
ReAct Loop
A Reasoning-Acting cycle where an agent thinks, takes an action, observes the result, and repeats. Deliberately excluded here because the problem's resource boundaries are known and the loop would add cost without value.
From the discussion
Featured comments
用户445507979014

AI Mind image generation failure also counts toward the limit. When it fails, it only says you need to add auxiliary words but doesn't specify how to supplement them. The result is that without a single image being produced, it becomes unusable, and it prompts that the three trial attempts have been used up.

倾颜

You can try generating a photo of a small animal. Clearing the cookie allows you to retry.

用户445507979014  → 倾颜

It always gets stuck here.

See top comments, translated →
Source: juejin.cn ↗ Google Translate ↗ Backup ↗