Agent Demos Are One Lucky Sample; Production Runs the Whole Distribution
Put two sets of numbers together and the gap appears. On LiveBench 2.0's general-tasks leaderboard, Claude Fable 5.2 completes 82.9 tasks and ChatGPT-6 Astra completes 75.6; in the simulated enterprise environment built by AI infrastructure company EON, the best performer, Claude Opus 4.8, only reaches a 76.8% task success rate. Change the environment and the same batch of top models takes a hit on its report card. People who actually push agents into production know that one pass in a demo video and ten thousand runs without a failure in production are two completely different things. This gap is not a problem with one particular model; it is a structural disparity the entire agent industry faces together. Whoever understands this first saves real money.
The gap between demo and production is essentially a property of probabilistic systems
Why are demos always so good-looking? Because a demo is a deliberately selected single sample. An agent is a probabilistic system running in an unpredictable environment. Run the same task ten times and nine may be perfect while one goes off the rails. When recording, splice in the nine good runs, cut the failed one, and a perfect demo is born.
Production does not accept editing. A customer experiences your agent through the distribution of one million calls, and that one percent of aberrant output is ten thousand incidents every day. This is not a model-capability problem; it is a perspective problem in evaluation. A demo measures the best single attempt; production demands the worst behavior in the tail of the distribution.
Once this is clear, the goal of evaluation changes. It is no longer answering whether the agent is smart, but how bad it can get when nobody is watching. The answers to those two questions are often unrelated.
Intelligence determines the ceiling; reliability determines the floor. Today every public leaderboard measures the ceiling, and almost nobody measures the floor. That is the structural root of dazzling demos failing in production.
This gap keeps recurring because of incentives: demos drive funding and sales, production determines retention and reputation, and the two run on completely different timelines. Fundraising runs on quarters, production incidents run by the hour, and teams naturally optimize the former. Only when customers renew based on results does tail reliability actually become a revenue variable.
Until this is recognized, any discussion of agent reliability engineering will be led astray by pretty demo data and land on no executable improvement.
Three typical ways agents fail in production
TechCrunch's November 11 report lists three recurring failure patterns, each worth remembering on its own.
The first is strategy drift. An agent assigned to book business travel, when facing a client's soft constraints, chooses to ignore the budget policy. Nothing has broken; it has simply put task completion ahead of rule compliance. Nobody tests this in a demo, because there is no budget policy in a demo.
The second is multi-source information entanglement. Ask it to look up a database and it reliably returns the right answer. Ask it to stitch together information from three datasets to answer a question and the whole flow starts stalling. Single-point capability is real; combined capability is another matter, and production tasks are almost all combined tasks.
The third is long conversations crushing the context. In a customer-service scenario, the longer the conversation and the more information it holds, the more easily the agent wanders off at some inflection point. Demo conversations are short, production conversations are long, and the evaluation length and the real length differ by orders of magnitude.
What these three patterns share is that none of them falls inside the scope of standard leaderboard measurement. Leaderboards measure capability; production tests boundaries. An agent that scores high on a leaderboard can still reliably fail on budget policies, multi-source stitching, and long contexts.
Worse, the three failure types stack on each other. When a long conversation contains both multi-source stitching and policy constraints, the failure rate is multiplied, not added. Production teams' real experience is that fixing any single type is quick, while the combination becomes an unmeasurable, occult fault nobody can fully explain.
Digital twins: moving your company's systems into the evaluation environment
Plurai, a company that just took a $13 million seed round, directly copies the autonomous-driving playbook. Team8 led the round, with U&I, Velocity, and Mercer following. Its core move is to build a digital twin of your company's electronic systems: knowledge base, tools, and interfaces all replicated inside a simulation environment. The production environment the agent will face is moved wholesale into the evaluation arena.
With that replica, the next step is drawing all possible action paths of the agent as a graph. Every branch is a potential incident point; every intersection is a potential information-entanglement point. Evaluation is no longer random sampling of questions; it systematically walks every path on the graph.
The focus is boundary scenarios. Anyone can walk the normal paths; what truly distinguishes reliability are the low-frequency but fatal corners: a user suddenly changing requirements, an upstream interface timing out, two business rules conflicting. A digital twin's value is batch-producing those corners without waiting for real customers to step on them.
The platform is designed to coexist with existing tools; Claude Code, Codex, Copilot, and Cursor can all plug in. It does not build your agent; it builds the examination room for your agent.
The lesson for engineering teams is direct: the evaluation environment itself is infrastructure that needs to be built and invested in, not an afterthought. Without a simulation environment close to production, any pre-launch evaluation conclusion deserves a question mark.
The digital twin has another underrated function: it makes incidents replayable. Once a production incident happens, load the state at that moment into the twin and replay it to locate whether it was model drift, a data problem, or a process defect, instead of guessing from logs.
The autonomous-driving industry already paid this tuition
Plurai's two founders, Ilan Kadar and Elad Levi, both come from autonomous driving. Kadar was previously at Nexar, and before that collaborated with Waymo on simulation and evaluation of end-to-end driving software. Their reason for switching to agents, as Kadar said plainly himself: when he saw the wave of AI agents, he saw the same pattern. Making a demo is easy; making something both reliable and economical is hard.
The autonomous-driving industry took ten years to truly digest that sentence. Early autonomous-driving demos were equally stunning, flowing smoothly in closed lots and crashing constantly once on open roads. The path the industry finally found was not stacking bigger models but stacking simulation: synthetic data manufactures millions of boundary scenarios that reality would not present in years, and the system pays its tuition inside a virtual world.
The agent industry is re-walking the same road. Today's agent demos sit exactly where autonomous-driving demos once sat, and the answer is most likely the same: simulation environments, synthetic boundaries, systematic evaluation. The difference is that an agent testing ground is far cheaper than roads; one can be set up in an afternoon.
The size of this tuition is known, because someone else already paid it. Autonomous driving also taught evaluation a second lesson: mileage is meaningless; scenario coverage is what matters. A million miles of sunny highway are worth less than ten thousand rainy-night detours plus construction detours. Agent evaluation is the same: ten thousand smooth Q&A exchanges are less informative than a hundred carefully constructed adversarial scenarios. That cognitive shift took the autonomous-driving industry years and expensive costs to complete.
The choice is simply to copy it, or to pay for it yourself again.
A single success-rate number, by itself, says nothing
Seeing a figure like 76.8%, engineering intuition should immediately ask about sample size. 76 successes in 100 tasks, versus 7,680 in 10,000, are two wildly different strengths of evidence. The former can statistically anchor no conclusion at all.
The standard practice for judging such numbers is to give a confidence interval. For a binomial metric like task success rate, the Wilson interval is the common tool, computable in a few lines of code:
import math
def wilson_interval(k, n, z=1.96):
if n == 0:
return (0.0, 0.0)
p = k / n
d = 1 + z * z / n
center = (p + z * z / (2 * n)) / d
half = (z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n))) / d
return (center - half, center + half)
lo, hi = wilson_interval(76, 100)
print(f"{lo:.3f} ~ {hi:.3f}")
Run it and you will see: a 76.8% success rate on 100 tasks gives a 95% confidence interval so wide it frightens. Pull that interval into evaluation reports and many pretty scores are exposed at once.
The value of this habit goes beyond evaluation itself. Once you require every success-rate report to carry sample size and interval, vendors' demo talk dies on its own. Numbers can no longer bully the discussion; the discussion is forced back onto evidence strength.
What does a wide interval mean? It means you cannot distinguish a 70% vendor from an 82% vendor, because their confidence intervals overlap heavily, and choosing either is a coin flip. It also means a single demo run showing 90% success may carry an interval from 70% all the way to 98%, from which no conclusion can be drawn. To shrink the interval to a decision-useful width, the sample size has to reach the hundreds, which is exactly why systematically generated test cases become a necessity; hand-crafting hundreds of representative boundary scenarios is not feasible.
One further practical rule: the closer a success rate is to 50%, the more samples are needed; the closer to perfect, the less you can rely on the success rate alone and the more you should count the types of failures. A hundred tests with zero failures proves nothing about reliability; it only proves your test set is not hard enough yet.
Beyond the score: the business it actually wants to sell
Evaluation is only the entry point. Plurai's real ambition sits further back: using the data collected through simulation to post-train, for each customer, a dedicated small open-source model. Official tests in cooperation with Nvidia and Nebius show these custom small models outperforming mid-tier frontier models.
The business logic is smooth. Build the simulation environment, accumulate boundary-scenario data, and distill the general model into a small model fitted to your business. Cost drops a tier, reliability rises a tier, and the data stays in your hands.
For customers this means a shift: model selection is no longer buying the biggest and most expensive, but feeding your own failure cases into a model that knows your business best. Evaluation data goes from quality-inspection report to training asset, and evaluation spending goes from cost center to investment.
This is the segment of the whole chain most worth watching. If post-trained small models are validated, the ledger of agent economics gets entirely recalculated, because production never wanted the smartest model; it wanted the model that knows this production line best.
This route carries even more reference value for cost-sensitive teams. Open-source small models are already cheap, and the post-training data comes from your own simulation environment, depending on no vendor's private data. While general large-model API calls keep burning money by usage, a small model post-trained for your own process is a fixed asset, one investment amortized over the long term. When evaluation, simulation, and post-training chain together, the cost structure moves from variable to depreciation. That is the scalable business shape.
Minimum moves: giving your own agent a health check
Compress the above into executable actions and only three steps remain. First, pick twenty of the most frequently used task flows from your real business and write them into a repeatable test suite. Second, deliberately inject three kinds of boundaries: budget or permission constraints, multi-data-source combinations, and over-long contexts. Third, report success rates with confidence intervals.
| Evaluation mode | Environment | What it measures | What it misses |
|---|---|---|---|
| Public leaderboard | Generic task set | Model's general capability ceiling | Your business boundaries |
| Simulated enterprise environment | Simulated generic company | Rough performance on combined tasks | Your specific rules and data |
| Self-built digital twin | Copy of your own systems | Pass rate on real edge-case scenarios | Nothing; this is the final form |
The three evaluation modes are not alternatives; they are progressive. Leaderboards are for initial screening of candidate models, simulated environments for horizontal comparison, and the digital twin for the final verdict before launch. Skipping the last two and making production decisions directly from leaderboard numbers is operating on yourself using somebody else's physical exam report.
The window to act is now. The evaluation toolchain has just taken shape; whoever builds a digital twin of their own system a step earlier will reach a reliable agent faster than whoever chases model version numbers. Models change every three months, while your business process does not change for years. Building evaluation assets on the process is the investment that only rises. Do not aim for perfection in one pass: in the first week run one baseline across twenty flows, in the second week add boundary scenarios. Rolling iteration is more realistic than filling everything at once.