Agent Reliability Is a Distribution, Not a Score

Every agent announcement arrives with a number, and the number is almost always a single pass. The system ran the task once under benchmark conditions and finished it. That result then gets read as a reliability claim, which is not what it is.

Reliability is a statistical property of a system that runs repeatedly under changing conditions, and one run cannot measure it. What follows is why a single score understates the spread, why benchmark infrastructure itself turned out to be gameable, and what to measure instead.

Why can you not trust a single score?

Because a single score is an existence proof, not a quality gate. It shows the agent completed the task once under controlled conditions. It says nothing about how often it succeeds on a real workload, where inputs, latency, and state change on every run.

"Towards a Science of AI Agent Reliability" opens on exactly this gap: "While rising accuracy scores on standard benchmarks suggest rapid progress, many agents still continue to fail in practice." The authors name the cause as a limitation of the measurement rather than of the models, arguing that "compressing agent behavior into a single success metric obscures critical operational flaws." Current evaluations, they write, ignore whether agents behave consistently across runs, withstand perturbations, fail predictably, or have bounded error severity.

Why does the same agent give different results?

Because almost nothing in the stack is deterministic. Model sampling is stochastic, tool call ordering is not guaranteed, and retries take different paths through the same task. Run the same agent on the same input twice and you can get two different outcomes without changing anything.

That makes reliability a distribution with a mean, a variance, and a worst case. A task that passes on the first attempt may fail on the third, and the first run is the least informative one to report. This is also why the control loop belongs inside the system under test: an agent without iteration caps and timeouts spends its variance on runaway behaviour rather than on finishing the job. Measure the loop and the model together, never the model alone. Making the loop itself reliable is a separate problem from measuring it.

What did the adversarial stress test of agent benchmarks find?

That the benchmarks themselves could be beaten without doing the work. In April 2026, a UC Berkeley RDI team (Hao Wang, Qiuyang Mang, Alvin Cheung, Koushik Sen, and Dawn Song) stress-tested eight leading agent benchmarks and reported that every single one "can be exploited to achieve near-perfect scores without solving a single task."

None of the exploits involved improving a model. Each one attacked what the benchmark was able to observe.

OSWorld is the interesting row. It scored lowest, but not because it defended itself. It leaked its answers through a different channel and the researchers attributed the gap to partial isolation rather than to defensive design. A benchmark that is harder to cheat by accident is not the same as one built to resist cheating.

Why can a benchmark be gamed without solving a task?

Because a benchmark measures only what it can observe. When the observation channel is reachable by the thing being measured, the score stops reporting capability and starts reporting the exploit.

In WebArena, the agent did not shop or browse. It read the stored answer out of a config file through a file:// URL and submitted it, and the evaluator could not distinguish a correct outcome from a copied one. That is the general mechanism: a score is a measurement of its own observation channel as much as of the agent. When the channel leaks, a high number is evidence that a result appeared, not that a correct process produced it.

Does the model or the scaffolding move the number more?

Usually the scaffolding. Tool definitions, retrieval, prompts, and the control loop determine how reliably the system performs, and the differences between model generations are frequently smaller than the differences these make. When a result changes between two setups, isolate whether the model or the surrounding code changed before you attribute the gain.

This is why "an agent is a system, not a model" is a practical instruction rather than a slogan. The harness around the model is part of what you are testing, and so is how many tools you put in front of it. Benchmarking a checkpoint in isolation tells you very little about the system you actually deploy.

What should you measure instead of a single score?

The distribution. Run the same task K times and record the pass rate, the variance, the worst case, and how the result drifts over time. pass@k answers the capability question, whether the agent can ever solve the task. The repeated-run pass rate answers the reliability question, how often it does. Those are different numbers and they get confused constantly. The gap between them is not academic: sampling 250 times took SWE-bench Lite from 15.9% to 56% — coverage, not delivered accuracy.

The reliability paper proposes twelve concrete metrics across four dimensions, which is a more useful checklist than any single figure:

• Consistency. Does it behave the same way across repeated runs?
• Robustness. Does it hold up when the inputs are perturbed?
• Predictability. When it fails, does it fail in ways you can anticipate?
• Safety. Is the severity of its errors bounded?

Report the shape, not just the point. A mean with no spread hides a system that succeeds nine times and then fails hard, and the worst case is what a deployment decision actually depends on.

What is a practical evaluation protocol?

Fix the task and its inputs. Run the agent repeatedly, enough trials to see the spread, recording pass rate, variance, worst case, and latency each time. Repeat under edge-case and adversarial inputs, which tests the surrounding code rather than the model. Track the result over time to catch drift, and review the plan before the agent spends compute.

A reliability figure without a trial count is an anecdote. Run enough trials to separate signal from variance, and publish the distribution alongside the protocol that produced it.

How should you read a vendor's benchmark claim?

Ask three questions: how many trials, which benchmark, and how was the outcome observed. A single score is an upper bound on capability. A vendor reporting a distribution over runs with a trial count is telling you considerably more than one reporting a single number, and the Berkeley result means the third question is no longer pedantic.

Apply the same test to any agent you deploy regardless of where it runs, including ones you rent rather than build. Askpert sells access to agents built by domain experts, and a rented agent deserves the same repeated-run scrutiny as one you wrote yourself. The standard does not change because someone else maintains the system.