Half of enterprises shipped a failing agent
That number should stop anyone building AI agents for customers cold. According to new VentureBeat Pulse Research surveying 157 enterprise organizations, 50% have deployed an agent or LLM feature that passed internal evaluations — and then caused a customer-facing failure in production. A quarter have seen it happen more than once.
The finding lands like a punch. It means the standard pre-deployment gauntlet — unit tests, red-teaming, automated evals — is letting bad agents through. The test says go. The agent breaks. The customer pays.
Only 36% of organizations report no such failure. The rest either don’t run pre-deployment evaluations at all (8%) or don’t track root causes closely enough to know (6%).
Trust in automated evaluation is almost nonexistent
Ask enterprise leaders how much they trust automated evaluation today, and the answer is brutal: just 5% say they fully trust it. That leaves 95% with a specific complaint holding them back.
The top grievance, cited by 29% of respondents, is the one that explains the failure rate: evaluations align poorly with real-world outcomes. A passing score in the test environment doesn’t predict what happens when real users, real data, and real edge cases show up.
Bias and inconsistency (21%) come next, followed by lack of explainability (18%) — organizations can’t always understand why an evaluation reached its verdict. Another 17% cite data leakage or privacy concerns in the evaluation process itself.
So the tests meant to certify agents are, broadly speaking, not trusted to certify them. That makes what comes next all the more surprising.
Autonomy is accelerating — despite the trust gap
Here’s the paradox at the heart of the research. Even though almost no one fully trusts automated evaluation, two-thirds of organizations (66%) either already allow zero-human-in-the-loop deployment for low-risk agents (34%) or are actively engineering their pipelines to permit it within twelve months (33%).
Only 22% rule it out for the foreseeable future.
The direction is clear: enterprises are moving to let evaluations gate production autonomously, removing the human check, at the same moment they say those evaluations don’t reliably match reality. The autonomy ceiling is rising faster than the assurance beneath it.
Notably, this isn’t just a startup phenomenon. Larger enterprises (2,500+ employees) are slightly more likely than smaller ones to be on the zero-human-review path (70% versus 64%) and slightly more likely to have shipped a failing agent (54% versus 48%). The assumption that big, regulated organizations hold the human in the loop longest is, in this sample, backwards.
The evaluation stack is fragmented — and provider-led
Ask which agent reliability or evaluation platform enterprises primarily use, and the market has no clear leader. Provider-native tooling leads: OpenAI‘s native evals and traces (17%) and Anthropic‘s Claude Console evals (13%) together outweigh any independent platform.
But they’re tied at the top by a striking answer: 17% of enterprises use no dedicated agent-evaluation tooling at all.
The specialist vendors — DeepEval (12%), Braintrust (8%), LangSmith, Weave, Promptfoo, Langfuse, Arize — are scattered across single to low double digits. Another 11% have built their own. No independent platform has yet become the category standard.
Production monitoring mostly watches uptime, not correctness
Production monitoring for an AI agent can watch two very different things. It can watch whether the system is functioning — is the agent up, how fast, at what cost, any errors. Or it can watch whether the agent’s output is correct — automated checks on each answer’s content.
The distinction matters because a confidently wrong answer is invisible to the first kind of monitoring. The request completes. The response is fast. No error is thrown. Everything reads healthy.
The split is stark: 51% of organizations monitor only whether the agent is functioning, while just 23% monitor whether its answers are right. Roughly three-quarters run no automated, real-time evaluation of output correctness in production. They’re taking correctness on faith.
What drives tool selection — and what’s next
Enterprises buy evaluation tooling on economics and trust it on repeatability. Cost of evaluations (28%) narrowly leads selection, just ahead of ease of integration (27%) and evaluation accuracy (24%). Breadth of observability (13%) and vendor roadmap (4%) matter far less.
On what success looks like, more than a third (36%) name evaluation consistency — getting the same verdict on the same behavior every time. That’s well ahead of speed of experimentation (19%), reduction in failures (18%), production visibility (13%), and compliance (11%).
The emphasis on consistency is telling: before enterprises can trust an evaluation’s verdict, they need it to be stable — the very property whose absence (bias and inconsistency) ranked among the top trust limitations.
A tooling reshuffle is coming
The evaluation market is wide open. While 36% have no plans to change, a clear majority (64%) intend to adopt a new, additional, or replacement platform within twelve months. 31% plan to do so within the next quarter.
The consideration set points where current usage is thinnest: DeepEval leads what enterprises are evaluating (20%), ahead of OpenAI’s native evals (13%) and Braintrust (9%). The open-source specialists are drawing more interest than their present footprint suggests.
Given that so many enterprises today rely on provider-native tools or nothing at all, this is less a defection than a first real wave of tooling adoption — the moment the evaluation layer starts to consolidate.
The bottom line: An evaluation gap that autonomy will widen
Organizations with 100 or more employees are granting AI agents more independence than they trust their evaluations to support. Half have already shipped an agent that passed its evals and then failed a customer. Almost none fully trust automated evaluation, chiefly because it doesn’t match real-world outcomes. Most watch production for uptime and cost rather than for whether the agent’s answers are right.
Yet two-thirds already allow, or are actively building toward, deploying to production on automated evaluation alone.
The vendor market is early and unsettled. Encouragingly, the next dollar is going to observability and — pointedly — human review, suggesting enterprises sense the gap even as they engineer past it. At 157 respondents in a single wave this is a directional read, skewed toward the mid-market. But the direction is clear: autonomy is being granted on the strength of evaluations that the people granting it do not yet trust.
The evaluation gap is not a coverage problem that more tests alone will close. It is a problem of evaluations that reflect reality and can be trusted to gate it. The open question for later waves is whether assurance catches up to autonomy — or whether the false-confidence failures move from customer incidents into changes that deploy themselves.
CyberSecurity6 months ago
How To5 months ago
CyberSecurity6 months ago
Social Media6 months ago
Video4 months ago
Infosecurity6 months ago
CyberSecurity6 months ago