Agent Output Quality Depends on the Harness, Not Just the Model
Teams chasing better agent results by swapping models are optimizing the wrong variable. The same model produces order-of-magnitude differences in output quality depending on how tasks are defined, context is managed, and human checkpoints are placed — and those practices are cheap to adopt compared to model upgrades.
Two people using the same agent can get wildly different results. One gets hollow, formatted text that needs endless correction; the other gets output that increasingly matches real work needs. The difference is the Harness — the working environment built around the model. Task clarity, context curation, tool permissions, human intervention points, result verification, and failure handling collectively determine whether a model's capabilities translate into stable, useful output.
A vague instruction like "analyze the market" is an idea, not a task. An agent will guess, and guess wrong with confidence. Specifying background, objective, expected result, boundaries, and acceptance criteria — and requiring every judgment to cite source evidence — prevents fabrication. Context is not about dumping documents; it's about showing the right information at the right time, including persisting intermediate state so long-running tasks can recover from drift.
Human-agent division shouldn't happen at the job-role level but at the task-unit level. Clear, low-risk, reversible tasks can run autonomously; fuzzy, high-stakes, or irreversible operations need human control. And a single successful run isn't enough — the prompt, context, rules, workflow, and known failure modes should be packaged into a reusable capability that gets retested on a different instance to avoid overfitting to one case.
Framing agent reliability as a Harness problem rather than a model problem shifts the optimization target from expensive model swaps to cheap process design, which most teams underinvest in.
The advice to persist intermediate state during long agent runs mirrors checkpointing in distributed systems — a pattern that agent workflows have not yet standardized on but clearly need.
Requiring source citations for every agent judgment is a lightweight guardrail that costs almost nothing to implement yet directly attacks the hallucination problem at the output layer rather than the model layer.
The overfitting analogy for agent workflows is sharp: a prompt that works once may have succeeded due to accidental context, and without retesting on a different instance, teams are just accumulating brittle templates.