Enterprises are spending big on AI infrastructure — but they can barely see where the money goes

New research from VentureBeat paints a stark picture: the vast majority of enterprises are pouring money into AI compute while flying nearly blind on cost. The report, drawn from a Q2 2026 survey of 107 organizations with over 100 employees, identifies what it calls an AI compute gap — the widening distance between how aggressively companies invest in AI hardware and how poorly they track its economics.

Only 21% of respondents run AI in production at scale. Yet spending intentions are already racing ahead of that maturity. The single largest area enterprises plan to evaluate over the next year is AI-specialized clouds (45%) — a category almost none of them use today. Meanwhile, the compute they already own sits mostly idle: 83% report GPU utilization of 50% or less. Fewer than half (44%) rigorously track what their AI compute actually costs.

“Enterprises are buying more infrastructure faster than they can account for what they already own,” the report states. That gap is the central tension of the moment.

GPU utilization is abysmal — and largely unmeasured

Perhaps the most striking number in the study: 83% of enterprises that operate GPUs report utilization at or below 50%. Nearly half (49%) run at 25% or below. Only 12% clear the 50% mark. Another 8% don’t measure utilization at all.

Idle accelerators are expensive accelerators. A single Nvidia H100 can cost tens of thousands of dollars. Let whole clusters sit half-empty, and the waste compounds fast. The report calls this the clearest single measure of the compute gap: enterprises plan to buy more GPUs and specialized compute while the capacity they already own sits substantially unused.

The measurement problem runs deeper than utilization. Fewer than half of enterprises (44%) rigorously track the cost and return of their AI compute. Another 39% track only partially. Twenty percent cannot quantify it yet, and 6% have not prioritized it at all.

That’s a problem because total cost of ownership (TCO) is the second-most important factor when enterprises choose an AI infrastructure provider — cited by 35% of respondents. Integration with the existing stack ranks first (41%). Headline price? Cost per million tokens matters to just 8%, dead last. “Enterprises are choosing providers on an economic basis they mostly cannot yet measure,” the report notes.

A switching wave is building — most within the year

Enterprises are not loyal to their current infrastructure vendors. A clear majority (64%) plan to switch or add an infrastructure provider within twelve months. Even more striking: 38% intend to do so within the next quarter. That is unusually high churn intent for a category as foundational as compute.

Where does that interest point? Mostly at the incumbents. Microsoft Azure and Google Cloud each draw 33% switching consideration, followed by OpenAI (30%) and Gemini (22%). The report suggests much of the near-term movement is reshuffling among the majors and consolidating spend — not defecting to new entrants. The neocloud interest is a 12-month evaluation thesis; the switching in the next quarter is mostly incumbents trading share.

The next dollar goes to infrastructure they don’t yet run

Here is the report’s sharpest tension. The single most-cited planned evaluation area — AI-specialized clouds, at 45% — is the very category almost none of these enterprises use today. The specialized “neocloud” GPU providers that dominate AI-infrastructure headlines — CoreWeave, Lambda, Crusoe, Nebius and peers — register at or near zero among these enterprises today.

Nearly a third (32%) intend to evaluate non-Nvidia accelerators. Twenty-eight percent plan to look at next-generation Nvidia silicon. Even decentralized compute networks (16%) and sovereign compute (11%) draw meaningful interest. Read against current usage, this is not incremental — it is the leading edge of a re-platforming.

The direction-of-travel question tells the same story: every infrastructure approach is net-expanding, but specialized AI clouds carry the highest net momentum (+24), edging out even the hyperscalers (+22). Enterprises are preparing to move a meaningful share of AI compute off the general-purpose cloud.

The next bottleneck: memory, not compute

The report also flags a frontier constraint that is barely on most enterprises’ radar. As large-scale inference scales, the binding constraint shifts from GPU compute to memory bandwidth — specifically KV-cache capacity. Asked how they would address this shift, enterprises scatter: Dell leads at 31%, Nvidia follows at 16%, and the rest fragments across storage vendors, open-source tooling, and model-level efficiency techniques.

Most telling: roughly one in five (18%) either do not recognize the constraint or have not begun to address it. “For a shift that will reshape inference cost and architecture, this is an early and unsettled market,” the report notes. It is the next chapter of the compute gap, arriving before most have closed the current one.

What this means for enterprise AI strategy

The report’s bottom line is blunt: the compute gap is not a capacity problem that more hardware will solve on its own. It is, first, a problem of seeing what the hardware already costs.

For enterprises, the implications are concrete. Before committing to specialized clouds or alternative accelerators, organizations should invest in instrumentation — utilization monitoring, cost allocation, TCO modeling. Without that visibility, the next round of spending risks repeating the same inefficiencies at a larger scale.

Satisfaction with current infrastructure is moderately positive (4.0 on a five-point scale) but softest on value for money — the dimension hardest to judge without measurement. That softness is a signal. Enterprises that build cost visibility now will be better positioned to evaluate the specialized clouds and alternative accelerators they plan to assess. Those that don’t will be buying the next layer of infrastructure as blind to its economics as the last.

The open question for later waves is whether enterprises build that visibility before the re-platforming arrives — or after.

Why hospital pharmacy teams spend 4,000 hours a year on a single compliance task

Every year, a single hospital covered under the federal 340B drug pricing program can burn more than 4,000 staff hours just checking whether Group Purchasing Organisation (GPO) drug purchases qualify for an exception. That is nearly two full-time employees dedicated to comparing purchase data against FDA shortage notices, ASHP records, inventory levels, machine-learning shortage forecasts, and back-order reports from other hospitals. It is manual, repetitive, and expensive.

Now Amazon Web Services and Bluesight say they have built an AI layer that can do most of that work in minutes. The product, called Prism, connects hospital pharmacy and compliance data across Bluesight’s existing suite of tools. Its first module, Prism Assistant for ControlCheck, has reached general availability and is already operating across 20 health systems.

A second, more ambitious agent—designed to handle full 340B GPO compliance—is scheduled for release later in 2026.

How Bluesight built Prism Assistant in three days

Bluesight started with ControlCheck, its controlled-substance monitoring product. Hospital diversion teams use it to spot unusual medication transaction patterns. But compliance staff still had to manually assemble reports, review dashboards, and correlate findings. That is where Prism Assistant comes in.

It offers a conversational interface that can query ControlCheck data, generate charts, and produce report material. AWS claims Bluesight built the first version during a three-day Experience-Based Acceleration engagement in September 2025. Eight Bluesight engineers worked alongside seven AWS specialists. While those rapid timelines highlight the agility of the tools, they remain vendor-reported metrics—independent verification from the active health systems is still pending.

The technical architecture is worth unpacking. The team used Strands Agents with Amazon Bedrock and hosted the application through Amazon Bedrock AgentCore Runtime. AgentCore Gateway exposed more than 10 ControlCheck APIs as MCP tools, allowing the agent to discover and call them during a user request.

Crucially, Bluesight avoided giving the language model direct database access. Instead, engineers wrapped existing ControlCheck API endpoints in AWS Lambda functions that return structured data suited to agent processing. Business logic stayed inside the application layer. The agent simply interpreted questions, selected tools, gathered records, and presented results.

AWS reports that design reduced query latency from five minutes to 10 seconds. The deployment also includes a frontend with chart generation, observability controls, cost attribution, encryption, authentication, and infrastructure-as-code.

“This is exactly what diversion program leaders have been waiting for—it gets them to answers faster and takes the manual grind out of every investigation,” said Samir Neyazi, Director of Product Management at Bluesight.

The 340B GPO compliance agent: multi-product orchestration

The bigger challenge is GPO compliance. Federal 340B rules prohibit Disproportionate Share, Children’s, and Free-Standing Cancer hospitals from buying outpatient drugs through GPO contracts when non-GPO channels can supply the drug. Compliance teams must document the exception when supply conditions prevent that purchase route.

Bluesight’s planned GPO agent brings together records from three products: CostCheck (purchase information), ShortageCheck (drug availability evidence), and 340BCheck (eligibility data). The proposed architecture uses Anthropic Claude Sonnet 4.6 as the primary model and Claude Haiku 4.5 for lower-latency operations, both running through Amazon Bedrock.

A coordinating GPO agent directs specialist data workers. One retrieves purchase records, another gathers supply evidence, and another checks 340B eligibility. The coordinator assembles the evidence and produces an audit-oriented report.

March 2026 brought a second AWS acceleration engagement focused on that architecture. AWS says the team connected the system by the end of its first day and completed every planned feature by day two. The company tested the agent against synthetic data, where it reported a 100 percent invoice discovery rate and 93 percent evidence justification accuracy—above its 85 percent target.

But enterprise buyers should exercise caution. Those figures do not represent production performance across hospital customers. Synthetic testing can demonstrate whether tool calls, matching logic, and report generation work against prepared scenarios. It cannot establish how the system handles local data gaps, delayed shortage updates, unusual drug identifiers, or disputed purchasing cases.

Why compliance scoring stays outside the language model

Bluesight assigns the language model a constrained role in the GPO workflow. The model gathers records, calls product tools, and drafts the explanation. A deterministic scoring service calculates the compliance determination.

That service evaluates 13 evidence inputs, applies priority-based matching, and uses configurable time windows. The design gives compliance teams a repeatable scoring process rather than an LLM-generated judgement. An auditor can inspect the source records, the rules applied, and the sequence of tool calls behind each determination.

Despite the automated assistance, hospital pharmacy, legal, and compliance teams still need absolute ownership of those policy settings. A supplier shortage threshold, acceptable inventory period, or purchase-date window can alter a compliance outcome. Bluesight’s approach gives customers a technical mechanism to configure those decisions, but each organisation must set and approve its own policy rules.

HIPAA controls, audit trails, and real-world performance

Amazon Bedrock holds HIPAA eligibility, and Bluesight operates under a Business Associate Agreement with AWS. AWS says it does not train foundation models on customer data processed through Amazon Bedrock.

Bluesight uses Amazon Cognito for OAuth2 client-credential authentication and JSON Web Token validation. AgentCore Runtime provides session isolation for concurrent customer requests. AWS Key Management Service encrypts data at rest and in transit, while AWS Secrets Manager manages credentials for downstream services.

Amazon CloudWatch records agent decisions, tool invocations, data-access events, alarms, and performance metrics. That audit trail matters when a hospital needs to explain why it permitted a GPO purchase or escalated a drug-diversion pattern.

Bluesight’s internal measurements across 20 health systems report up to 97 percent faster report generation and analysis in ControlCheck workflows. Recurring reports reportedly dropped from about six hours of manual assembly to 15 minutes—a 96 percent reduction. Pre-investigation triage dropped from three hours to about 10 minutes, while controlled-substance variance analysis fell from 30 minutes to less than one minute.

Teams should strictly run historical purchasing cases in parallel with existing review processes before allowing an agent-assisted result to affect compliance decisions. Local testing should rigorously examine data completeness, drug-code matching, shortage timing, exception rules, and cases where human reviewers previously disagreed. Each production finding should retain the scoring-rule version, source evidence, and tool trace that produced it.

For more on how AI is reshaping healthcare operations, see our coverage of AWS GraphRAG deployment cuts drug research cycles by 87% and AI for hospital pharmacy automation trends.

Voice AI startup Rime lands $24M to tackle enterprise call handling

San Francisco-based Rime has closed a $24 million Series A funding round, the company announced Wednesday. The round was led by M13 Ventures, with participation from Twilio Ventures, Corazon Capital, Unusual Ventures, and others. The startup builds voice AI models designed specifically for enterprise customer calls — an increasingly crowded space.

Founded in 2022 by former Stanford PhD student Lily Clifford, ex-Amazon Alexa engineer Brooke Larson, and Stanford engineer Ares Geovanos, Rime is taking a different approach from many rivals. Instead of scraping the web for audio data, the company built a recording studio in San Francisco to collect its own conversational data. That proprietary dataset, Clifford says, helps the models nail pronunciation of brand names and industry jargon without forcing clients to retrain models from scratch.

The problem with legacy IVR — and why AI still isn’t enough

Despite rapid advances in voice AI, Clifford is surprisingly blunt about the technology’s limits. Enterprises still lean heavily on legacy IVR systems, she told TechCrunch, because AI voice agents just aren’t good enough yet.

“The voice technology is still not there to automate the vast majority of enterprise phone calls,” Clifford said. “LLMs have made it a lot easier to build voice applications that work, but they haven’t changed how it feels to interact. Talking with a voice AI agent is not the most compelling experience for the end user. It’s kinda like a new IVR, but with a better voice.”

That honesty might seem unusual for a startup CEO pitching a voice AI product. But it also signals where Rime sees its edge: not in flashy demos, but in the gritty work of making models that actually sound natural on a call.

From three models to one: Rime’s shift to speech-to-speech

Rime initially used a pipeline of separate models for speech-to-text, text-to-speech, and a large language model. But the company is now pivoting toward a unified speech-to-speech architecture. The goal? Lower latency, better turn-taking, and handling real-world problems like background noise.

That shift also reduces the burden of orchestrating multiple models. Fewer moving parts means less complexity — and, ideally, more reliable performance. For enterprise clients in regulated industries like healthcare and finance, reliability matters more than buzzword compliance.

Who’s using Rime — and why they stay on the call longer

Rime claims its approach is already winning enterprise contracts. The company says it has customers in food service, healthcare, airlines, and fintech. Named clients include Mayo Clinic, Dialpad, Upstart, and Asurion.

The startup asserts that because of its training data and model design, customers stay on calls longer — a key metric for enterprise call centers. Longer calls can mean better issue resolution, higher satisfaction, and more upsell opportunities. That’s the kind of concrete outcome that wins budgets.

M13’s Morgan Blumberg, who is joining Rime’s board as part of the Series A, sees the company’s focus on technical fundamentals as a differentiator. “Companies like ElevenLabs have moved into being an orchestration and the application layer, going head to head with the Sierras and Decagons of the world,” Blumberg said. “I think there’s just so much more to be done technically, and Rime’s approach of pushing forward on the best model with low latency and high reliability in a regulated environment stands out.”

Hiring spree ahead: Rime plans to double down on R&D

With the fresh capital, Rime plans to expand its current team of 35 people. The company is hiring for model development, engineering, and partnerships. It recently brought on Rafael Valle, who worked on audio understanding at Meta Superintelligence Labs and NVIDIA’s applied deep learning audio research team, as Chief Scientist.

Rime had previously raised $5.5 million in a seed round last May. The new funding gives it a runway to compete in a market that includes ElevenLabs, Deepgram, Vapi, Retell, LiveKit, Decagon, and Sierra. But the startup is betting that its proprietary data and focus on regulated verticals will give it an edge that more generalist voice AI companies can’t easily replicate.

For now, Clifford and her team are banking on a simple thesis: enterprise call automation won’t be won by the fanciest demo, but by the model that sounds most human — and doesn’t make customers want to hang up.