Enterprises are spending big on AI infrastructure — but they can barely see where the money goes
New research from VentureBeat paints a stark picture: the vast majority of enterprises are pouring money into AI compute while flying nearly blind on cost. The report, drawn from a Q2 2026 survey of 107 organizations with over 100 employees, identifies what it calls an AI compute gap — the widening distance between how aggressively companies invest in AI hardware and how poorly they track its economics.
Only 21% of respondents run AI in production at scale. Yet spending intentions are already racing ahead of that maturity. The single largest area enterprises plan to evaluate over the next year is AI-specialized clouds (45%) — a category almost none of them use today. Meanwhile, the compute they already own sits mostly idle: 83% report GPU utilization of 50% or less. Fewer than half (44%) rigorously track what their AI compute actually costs.
“Enterprises are buying more infrastructure faster than they can account for what they already own,” the report states. That gap is the central tension of the moment.
GPU utilization is abysmal — and largely unmeasured
Perhaps the most striking number in the study: 83% of enterprises that operate GPUs report utilization at or below 50%. Nearly half (49%) run at 25% or below. Only 12% clear the 50% mark. Another 8% don’t measure utilization at all.
Idle accelerators are expensive accelerators. A single Nvidia H100 can cost tens of thousands of dollars. Let whole clusters sit half-empty, and the waste compounds fast. The report calls this the clearest single measure of the compute gap: enterprises plan to buy more GPUs and specialized compute while the capacity they already own sits substantially unused.
The measurement problem runs deeper than utilization. Fewer than half of enterprises (44%) rigorously track the cost and return of their AI compute. Another 39% track only partially. Twenty percent cannot quantify it yet, and 6% have not prioritized it at all.
That’s a problem because total cost of ownership (TCO) is the second-most important factor when enterprises choose an AI infrastructure provider — cited by 35% of respondents. Integration with the existing stack ranks first (41%). Headline price? Cost per million tokens matters to just 8%, dead last. “Enterprises are choosing providers on an economic basis they mostly cannot yet measure,” the report notes.
A switching wave is building — most within the year
Enterprises are not loyal to their current infrastructure vendors. A clear majority (64%) plan to switch or add an infrastructure provider within twelve months. Even more striking: 38% intend to do so within the next quarter. That is unusually high churn intent for a category as foundational as compute.
Where does that interest point? Mostly at the incumbents. Microsoft Azure and Google Cloud each draw 33% switching consideration, followed by OpenAI (30%) and Gemini (22%). The report suggests much of the near-term movement is reshuffling among the majors and consolidating spend — not defecting to new entrants. The neocloud interest is a 12-month evaluation thesis; the switching in the next quarter is mostly incumbents trading share.
The next dollar goes to infrastructure they don’t yet run
Here is the report’s sharpest tension. The single most-cited planned evaluation area — AI-specialized clouds, at 45% — is the very category almost none of these enterprises use today. The specialized “neocloud” GPU providers that dominate AI-infrastructure headlines — CoreWeave, Lambda, Crusoe, Nebius and peers — register at or near zero among these enterprises today.
Nearly a third (32%) intend to evaluate non-Nvidia accelerators. Twenty-eight percent plan to look at next-generation Nvidia silicon. Even decentralized compute networks (16%) and sovereign compute (11%) draw meaningful interest. Read against current usage, this is not incremental — it is the leading edge of a re-platforming.
The direction-of-travel question tells the same story: every infrastructure approach is net-expanding, but specialized AI clouds carry the highest net momentum (+24), edging out even the hyperscalers (+22). Enterprises are preparing to move a meaningful share of AI compute off the general-purpose cloud.
The next bottleneck: memory, not compute
The report also flags a frontier constraint that is barely on most enterprises’ radar. As large-scale inference scales, the binding constraint shifts from GPU compute to memory bandwidth — specifically KV-cache capacity. Asked how they would address this shift, enterprises scatter: Dell leads at 31%, Nvidia follows at 16%, and the rest fragments across storage vendors, open-source tooling, and model-level efficiency techniques.
Most telling: roughly one in five (18%) either do not recognize the constraint or have not begun to address it. “For a shift that will reshape inference cost and architecture, this is an early and unsettled market,” the report notes. It is the next chapter of the compute gap, arriving before most have closed the current one.
What this means for enterprise AI strategy
The report’s bottom line is blunt: the compute gap is not a capacity problem that more hardware will solve on its own. It is, first, a problem of seeing what the hardware already costs.
For enterprises, the implications are concrete. Before committing to specialized clouds or alternative accelerators, organizations should invest in instrumentation — utilization monitoring, cost allocation, TCO modeling. Without that visibility, the next round of spending risks repeating the same inefficiencies at a larger scale.
Satisfaction with current infrastructure is moderately positive (4.0 on a five-point scale) but softest on value for money — the dimension hardest to judge without measurement. That softness is a signal. Enterprises that build cost visibility now will be better positioned to evaluate the specialized clouds and alternative accelerators they plan to assess. Those that don’t will be buying the next layer of infrastructure as blind to its economics as the last.
The open question for later waves is whether enterprises build that visibility before the re-platforming arrives — or after.
