AI’s next big leap comes with a staggering electricity bill
The AI industry’s growing hunger for electricity has already alarmed utilities, governments, and tech giants. But a new study suggests the problem is about to get much, much worse — not with smarter chatbots, but with the rise of AI agents.
Researchers at the Korea Advanced Institute of Science and Technology (KAIST) have published what they call the first comprehensive analysis of the energy cost of AI agents — autonomous systems that reason, plan, and execute tasks without human hand-holding. Their conclusion? These systems can burn through up to 136.5 times more energy per query than a conventional generative AI model. That’s not a typo.
The paper, presented at the IEEE International Symposium on High-Performance Computer Architecture (HPCA) earlier this year, raises a blunt question: is the infrastructure behind tomorrow’s AI ready for what’s coming?
Why AI agents are so much more power-hungry
Standard chatbots like ChatGPT or Claude answer a prompt in one shot. They generate text, and they’re done. AI agents don’t work that way. They loop through multiple calls to large language models (LLMs), browse the web, execute code, fire up calculators, and talk to external software — all while solving a single complex task.
That makes them far more capable. It also makes them far more expensive.
The KAIST team, led by Professor Minsoo Rhu from the School of Electrical Engineering, treated AI agents as an entirely new category of data center workload. They measured real-world computational requirements. The numbers are sobering.
Response latency can spike by up to 153.7 times compared to standard chain-of-thought reasoning. And here’s the kicker: the expensive GPUs running these workloads sit idle up to 54.5 percent of the time, waiting for external tools to finish. The hardware keeps drawing power even when it’s not doing any actual AI computation. That’s a lot of wasted electricity.
348 watt-hours per query — and that’s just the start
To put a number on it: running an AI agent powered by a 70-billion-parameter language model — roughly the size of today’s commercial systems — required an average of 348.41 watt-hours per query. A conventional chatbot answering a simple question? About 136 times less.
The team then modeled a future scenario where AI agents handle 13.7 billion requests per day, roughly matching Google’s daily search traffic. Under that load, AI infrastructure would need about 198.9 gigawatts of electricity. That’s nearly half of the average power consumed by the entire United States. Today’s AI data centers can’t come close.
The hidden cost no one’s talking about
Companies like OpenAI, Google, Microsoft, Anthropic, and others are pouring billions into agentic AI, betting it’s the next big leap beyond conversational bots. But the study argues that better models alone won’t cut it anymore. Future progress depends just as much on more efficient semiconductors, smarter GPU utilization, better data-center design, and expanded power infrastructure.
Professor Rhu puts it plainly: AI competitiveness is shifting from building “smarter AI” to building more efficient AI. The team believes the path forward requires co-design — optimizing models, AI chips, servers, and energy systems together. Otherwise, operating costs spiral and sustainability goes out the window.
The researchers have open-sourced their AI agent benchmarks, hoping to push the industry toward tackling one of AI’s fastest-growing — and most overlooked — costs: electricity. Because if the next generation of AI is going to be this powerful, it had better learn to be efficient too.
