Making Multi-Agent Output Trustworthy with a Runtime Control Layer

Quick Glossary (so you can follow along on first read)

• Supervisor / Reviewer Agent / Plan Agent / Task Agent: These are the commander-in-chief, the fault-finder, the plan-writer, and the task-breaker, respectively. Collectively referred to as "Agents" below.
• Runtime: The system's "hardcore referee," making final decisions with hard-coded logic, unswayed by a model's wording.
• LLM: Refers to the underlying large language model, responsible for generating content and making judgments—but its output is not inherently stable.
• Contract and Schema: A contract is the prescribed "output format" for each agent; a schema is a machine-verifiable format convention—field types, allowed values, and lengths are all fixed, and undeclared fields are forbidden.

---

System Overview: One Collaboration Chain, One Core Problem

AI Mind's "Generate Delivery Plan" feature takes a requirement description as input and has multiple agents collaborate to produce a delivery plan (consisting of two parts: an actionable "Implementation Plan" and a decomposed "Task List"). The entire chain is fixed:

1. Supervisor first judges if the input is sufficient;
2. Plan Agent generates the implementation plan;
3. Task Agent breaks the plan down into specific tasks;
4. Three Reviewer Agents find issues in parallel from three angles: plan consistency, risk, and boundary;
5. If a review finds a problem that must be fixed, the corresponding plan or tasks undergo one round of revision;
6. Finally, a complete delivery report is generated.

In this chain, every link's output is generated by an LLM. And LLM output is naturally unstable—the same input, called twice, can produce results with different wording, different structure, or even different conclusions. When multiple agents collaborate in series, an upstream output becomes a downstream input, and this uncertainty is amplified layer by layer down the chain.

The core question this article discusses is: Under this premise, how do you design a set of mechanisms so that the output of multi-agent collaboration is trustworthy?

Version v0.4.11's answer is: establish a deterministic control layer, mastered by the runtime, between the agents' output and the system's decisions. This control layer consists of three progressive design decisions—first, make every agent's output machine-verifiable; second, ensure critical safety rules cannot be bypassed; third, make the feedback loop converge within a controllable scope.

---

Decision 1: Make Agent Output Machine-Verifiable

Problem

Agents output natural language, but embedded within that text are business conclusions—risk level, boundary status, whether a revision is needed. If you extract these conclusions using regex or keyword matching, a simple rephrasing by the agent can cause the system to make a different judgment.

For example: a risk review agent writes "overall risk is controllable" in the body text, versus "medium-to-high risk exists." These might map to different risk levels, but they could actually be expressing the same conclusion. Worse, if the review agent adds an extra explanatory sentence that happens to trigger a keyword match, the system will misjudge.

Solution

Each agent's output does not directly enter business logic. Instead, it first passes through a strict, role-specific schema validation. The validated structured data is the sole source of truth for system decisions. The Markdown body is preserved, but only for display; it does not participate in any machine judgment.

In the code, this design is jointly implemented by two modules: agent-contracts.ts (agent contract definitions) and contract-invocation.ts (contract invocation boundaries).

Key Implementation

First, the schema makes no compromises. Every role's schema uses .strict() mode. If an agent's output contains a field not declared in the schema, the entire result is rejected—not silently deleted and continued. An unknown field means the agent has overstepped its role boundary, and silently deleting it would mask this overreach.

Take the structured output for a reviewer finding an issue as an example:

// agent-contracts.ts - Schema definition for a review finding
export const reviewFindingDraftSchema = z
    .object({
        description: boundedText(),           // Problem description
        evidence: boundedTextList(10),         // Evidence list, max 10 items
        findingType: z.enum(['issue', 'observation']),
        // Only 'issue' type triggers subsequent revision
        requirement: z.enum(['required', 'advisory']),
        // Only 'required' level affects the final status
        severity: z.enum(['blocker', 'high', 'medium', 'low', 'info']),
        suggestedAction: boundedText(),        // Suggested action
        targetArtifacts: z.array(revisionTargetSchema).min(1).max(2),
        // Indicates which artifact needs modification: plan or tasks
    })
    .strict()

The code above uses Zod (a common schema validation library) to describe the format. Don't get hung up on the specific syntax—just grasp the key point: every field's type, allowed values, and length are fixed, and undeclared fields are forbidden.

Note the findingType and requirement fields. These aren't arbitrary enums—findingType distinguishes between "a problem that needs handling" and "an observation for reference only," while requirement distinguishes between "must fix" and "suggested fix." These two fields directly determine whether a revision is triggered later and what the final status will be. If this information were hidden in Markdown and guessed at via keywords, any change in phrasing could lead to a misjudgment.

Second, one fix, not infinite retries. The invokeBusinessAgentContract function in contract-invocation.ts implements this logic: on the first validation failure, the model gets one chance to self-correct, but the feedback only contains a {path, code} summary (e.g., ["findings.0.severity", "invalid_enum_value"]), without exposing the raw output. If it fails a second time, it terminates as a stage failure and stops trying. Two failures indicate a capability problem, not a format problem, making further retries pointless.

Third, generation and encoding use different models. Business judgments are made by the user-selected model ("What should the risk level of this requirement be?"), while structured encoding is fixed to deepseek/deepseek-v4-pro. The two have different responsibilities and reliability constraints—the business model handles creativity and judgment, the encoding model handles strict format adherence. A user switching business models will not affect the stability of the system's judgments.

Trade-offs

Each agent stage adds an extra model call for structured encoding, trading cost for reliability. .strict()'s rigid rejection means a typo in a legitimate field will cause the entire result to fail—but this is intentional. For a multi-agent collaboration system, "silently accepting an unexpected field and then making a wrong judgment" is far more dangerous than "failing safely and letting the user retry."

---

Decision 2: Write Safety Rules in Code, Not in Prompts

Problem

This system has some absolutely inviolable rules: the three reviewer agents (plan consistency, risk, boundary) must each execute exactly once, no more, no less; if any reviewer agent judges a block, the entire delivery chain must stop and cannot be overridden by a "pass" conclusion from another agent; the Supervisor cannot skip the plan or task stages and jump directly to review.

If these rules are written in prompts—"Please be sure to call the three reviewer agents," "Please respect the risk reviewer agent's block conclusion"—the agent can ignore them, misunderstand them, or be induced to bypass them by carefully crafted input. Prompts are soft constraints, but safety rules require hard guarantees.

Solution

Move all safety rules from the prompt level into the runtime's hard-coded logic. Agents are only responsible for "suggestions"; the runtime is responsible for "verdicts." In the code, this design is jointly implemented by two modules: delegation-policy.ts (delegation policy) and report-synthesis.ts (report synthesis).

Key Implementation

First, exact set validation. The validateExactReviewerRoles function in delegation-policy.ts performs an exact count on the set of reviewer roles declared by the Supervisor before any reviewer agent is launched: exactly three roles, each appearing once. It does not fill in missing ones, delete extra ones, or normalize the order.

// delegation-policy.ts - Exact set validation
const REQUIRED_REVIEWER_ROLES = ['general', 'risk', 'boundary']

export function validateExactReviewerRoles(reviewerRoles) {
    if (reviewerRoles.length !== REQUIRED_REVIEWER_ROLES.length) {
        return { message: '...', summary: 'The set of reviewer roles declared by the Supervisor is incomplete.' }
    }
    const counts = new Map()
    for (const role of reviewerRoles) {
        counts.set(role, (counts.get(role) ?? 0) + 1)
    }
    if (REQUIRED_REVIEWER_ROLES.some(role => counts.get(role) !== 1)) {
        return { message: '...', summary: 'The set of reviewer roles declared by the Supervisor is incomplete or contains duplicates.' }
    }
    return null  // Validation passed
}

This function looks simple—a Map count, three checks. But its very existence is a design decision: the Supervisor declares a set of reviewer roles in a prompt, but the runtime does not trust this declaration and must re-validate it. The declaration is "intent"; the validation is "enforcement." If the Supervisor's declaration is illegal, the entire review dispatch is rejected, and none of the three reviewer agents are launched.

Second, a pure function state matrix. The resolveReviewBundleStatus function in report-synthesis.ts uses a pure function (same input always yields the same output, with zero randomness) to calculate the final state. Its inputs are only the structured coverage (whether each of the three reviewer agents executed successfully) and the structured review results; it does not depend on any LLM text.

// report-synthesis.ts - Deterministic state calculation
export function resolveReviewBundleStatus(bundle): RunStatus {
    const coverage = Object.values(bundle.coverage)
    const completed = coverage.filter(state => state === 'completed').length
    if (completed === 0) return 'failed'
    // All three reviewer agents failed to execute → System failure

    const hardBlocked =
        (general?.disposition === 'blocked') ||
        (risk?.severity === 'blocker') ||
        (boundary?.boundaryStatus === 'blocked')
    if (hardBlocked) return 'blocked'
    // Any hard block → Blocked

    if (completed !== 3) return 'needs_review'
    // Incomplete coverage → Needs human review

    if (general?.disposition === 'needs_changes' ||
        general?.planTaskAlignment === 'misaligned' ||
        bundle.findings.some(f => f.findingType === 'issue' && f.requirement === 'required'))
        return 'needs_changes'
    // Has a required fix → Needs changes

    return 'pass'
}

The priority chain is clear and immutable: Execution Failure > Hard Block > Incomplete Coverage > Required Fix Exists > Pass. This function reads no Markdown, calls no regex, and depends on no model output. The same input always produces the same output.

Third, hard rules cannot be overridden. When the risk reviewer agent returns blocker (a fatal blocking level), no matter what conclusions the other reviewer agents give, the system state must be blocked. The same applies for a boundary reviewer agent block, and for a plan consistency reviewer agent block. The Supervisor cannot downgrade this conclusion, the report generation process cannot ignore it, and the output of any subsequent agent cannot override it.

Trade-offs

Hard-coded gates mean limited flexibility—you can't dynamically decide to "skip one boundary check today." But the integrity of a safety gate shouldn't be "flexible." In this design, we chose "deterministic but constrained" over "flexible but unverifiable."

---

Decision 3: Convergence Control for the Feedback Loop

Problem

Issues found during review need to be fixed—this is the core value of multi-agent collaboration. But here comes the problem: who decides what to fix? After the fix, who judges if it was done correctly? If you review again after fixing, and fix again after reviewing, where does this loop stop?

Suppose you let the Supervisor decide "what to fix"—the Supervisor understands the review comments, then tells the Plan Agent and Task Agent how to change things. This seems reasonable, but the Supervisor is itself an LLM; it might misunderstand the review comments, miss critical issues, or stuff unrelated things into the revision scope. If you then have the three reviewer agents re-review the revision, they'll find new problems, leading to another revision, then another review... the number of cycles and the cost become uncontrollable.

Solution

Revision targets are derived directly by the runtime from the validated, structured review findings, without relying on the Supervisor's judgment. There is at most one round of revision, and no second round of review is performed after it; the final judgment is handed back to the human.

In the code, this design is jointly implemented by the derivePostReviewDecision function in structured-delivery-manager.ts and the revision orchestration logic.

Key Implementation

First, revisions are derived by the runtime, not decided by the Supervisor. The core logic of the derivePostReviewDecision function is: filter the validated review findings for entries where findingType === 'issue' and requirement === 'required', then group them by each entry's targetArtifacts field (which indicates whether the plan or tasks need modification) to generate corresponding revision requests.

// structured-delivery-manager.ts - Runtime derives revision decision
export function derivePostReviewDecision(bundle, guidance?) {
    const actionableFindings = bundle.findings.filter(
        f => f.findingType === 'issue' && f.requirement === 'required'
    )
    if (actionableFindings.length === 0) return { action: 'finalize' }
    // No required fixes → Finalize directly

    const requests = (['plan', 'tasks']).flatMap(target => {
        const findings = actionableFindings.filter(
            f => f.targetArtifacts.includes(target)
        )
        if (findings.length === 0) return []
        return [{
            requestKey: `runtime-${target}-revision`,
            sourceFindingIds: findings.map(f => f.findingId),
            targets: [target],
            // The Supervisor's guidance only affects the explanatory text, not the action
        }]
    })
    return { action: 'revise', requests, revisionTargets: requests.map(r => r.targets[0]) }
}

The Supervisor provides an optional guidance (revision instructions) after the review, but it can only affect the requiredActions and summary text content within the revision request; it cannot change the "whether to revise" and "what to revise" decisions. These two decisions are derived entirely by the runtime from structured data—no required fixes means no revision; a finding saying to fix the plan means only the plan is fixed; a finding saying to fix the tasks means only the tasks are fixed; if both are involved, the plan is fixed first, and the tasks are then aligned.

Second, one revision, no re-review. In the revision path, the plan is executed before the tasks (maintaining dependency order), revision artifacts retain stable identifiers and incrementing version numbers, and the revision outcome (RevisionOutcome) traces back to the original review finding via findingId, but does not declare "problem resolved." Most critically, the termination logic after revision:

// structured-delivery-manager.ts - Hard-coded termination after revision
// This version ends after one controlled revision; without an independent re-review, it must not claim a pass.
runStatus = 'needs_review'

The system will not launch a second round of review, nor will it produce a second revision. The internal state is fixed to needs_review (requires human review) and will not be marked as pass.

Third, hand the final judgment back to the human. In the final report, the next step after revision is always "Please manually confirm the plan and tasks after this revision." The system is responsible for discovering problems, deriving revisions, and executing modifications, but it is not responsible for judging whether the modifications are correct. That judgment is handed back to the human.

Trade-offs

This is the most core trade-off in the entire system design. We acknowledge two things: first, LLM self-evaluation is unreliable—having a reviewer agent re-review the artifact it just revised is no different in nature from having a student grade their own homework; second, automatic loops are uncontrollable—the convergence conditions for multiple rounds of "review → revise → review" are hard to define, and the cost is unpredictable.

So the system's positioning is: The system is responsible for "fixing," and the human is responsible for "judging if the fix is right." This is not a missing feature, but a sober recognition of the system's boundary.

---

Current Boundaries: Things Deliberately Not Done

This version explicitly defines its capability boundaries. The following things are deliberately not touched:

• No ReAct (Reasoning-Acting) loops. Worker nodes have no exploration tools; the problem and resource boundaries are known. Introducing a "think-act-observe" loop would only increase call costs and uncontrollable states without producing the value needed for this version.
• No multi-round "review → revise → review" loops. At most one revision, and no second round of review after it. Having an LLM repeatedly review its own output is a loop that cannot converge.
• No generic DAG (Directed Acyclic Graph) scheduler. The topology is fixed: Supervisor → Plan Agent → Task Agent → three Reviewer Agents in parallel → optional revision. No dynamic dependency graphs or arbitrary parallel groups are introduced.
• No persistence. All dispatch plans, artifacts, review bundles, and review findings exist only within a single run and are discarded when the run ends.
• No final LLM polish. The final report is deterministically rendered from validated structured data, without relying on an extra model call to "beautify" it.

These boundaries are not "not done yet," but rather "decided against after judgment." They define the system's safety radius and also the directions that the next version can expand into.

---

Summary

Returning to the initial question: when multiple agents collaborate, how do you guarantee the final result is trustworthy?

Version v0.4.11's answer is not "make the model more accurate," but "make the system more reliable." The three layers each bear different responsibilities:

• Structured trust ensures every agent's conclusion is machine-verifiable and won't be misjudged due to phrasing changes;
• Hard gates ensure safety rules are not bypassed, and an LLM's suggestion does not become the system's verdict;
• Bounded feedback ensures the feedback loop converges within a controllable scope, without falling into an unreliable cycle of "auto-fix → auto-re-review."

These three design decisions take control back from the LLM and hand it to the runtime's deterministic logic. The LLM remains a collaborator—it is responsible for generating content, making judgments, and providing suggestions—but the final decision-making power rests with the runtime.

To validate this design, we ran three baseline comparisons using the same fixed evaluation sample set: ① a single agent generating directly (no collaboration at all); ② fixed multi-agent, but without looping back to modify after a review found issues; ③ v0.4.11's controlled feedback loop (the solution in this article). The results proved that structured contracts and hard gates, with a controllable cost increase, significantly improved the consistency of state judgments and the reliability of safety rules. This is not a perfect system, but its reliability is verifiable.

Future versions will continue to expand the agents' boundaries—but the starting point is the deterministic control layer already in place. Only when the runtime can reliably manage the output boundaries of agents can we dare to let agents do more.

---

Project Links

👉 GitHub: https://github.com/HWYD/ai-mind

👉 Live Demo: https://ai.hwyblog.cloud/instant-mind

If this article or the AI Mind project has been helpful to you, you're also welcome to give the project a Star⭐. This support is very important to me and will further motivate me to continue organizing the implementation process, design trade-offs, and post-mortem reviews of subsequent versions.