Jensen Huang’s warning for engineers who don’t use enough AI
Nvidia CEO Jensen Huang has a blunt metric for judging whether an engineer earns their keep: token consumption. Speaking on the All-In Podcast at the close of GTC 2026, Huang said if a $500,000 engineer’s annual AI token usage falls below half their salary, “I am going to be deeply alarmed.” The company is targeting a $2 billion yearly token bill for its engineering force.
That stark math reflects a shift already underway across corporate America. Money that once went to salaries is flowing to API calls. The four largest hyperscalers have guided roughly $700 billion in combined 2026 capital expenditure — nearly double last year. Meanwhile, outplacement firm Challenger, Gray & Christmas reports AI is the most-cited reason for US job cuts for a record fourth consecutive month.
An internal Meta memo obtained by Reuters described May’s elimination of 8,000 roles as necessary to offset the company’s massive investments, even as revenue grew 33% that quarter. These aren’t survival layoffs. They’re financing decisions.
But there’s a problem: the financing hasn’t delivered returns. Gartner surveyed 350 executives at companies with over $1 billion in revenue, all deploying AI agents or automation. Roughly 80% had cut headcount with no correlation to improved returns. Analyst Helen Poitevin’s verdict was blunt: “Workforce reductions may create budget room, but they do not create return.”
Uber learned the token side of that lesson the hard way. In December, the company gave 5,000 engineers AI coding tools. By April, it had exhausted its entire 2026 AI budget. Chief Operating Officer Andrew Macdonald admitted that despite 70% of committed code being AI-generated, the connection to anything customers notice is missing: “That link is not there yet.”
Put those two failures side by side and the real problem emerges. Companies treated the token bill as fixed and the workforce as flexible. The opposite is true. Payroll cuts happen once and take institutional knowledge with them. A token budget, it turns out, bends in half a dozen places — if anyone bothers to engineer it.
Where the token budget bends
The cheapest fix is also the least glamorous: stop paying to process the same text repeatedly. Prompt caching, now standard across major API providers, cuts the cost of repeated input by up to 90% under Anthropic’s and OpenAI’s published pricing. Static content like system instructions and reference documents gets processed once and reread at a fraction of the rate.
Security firm ProjectDiscovery documented raising its cache hit rate from 7% to 84% by restructuring prompts. That single engineering exercise cut total LLM spend by 59% to 70% while serving 9.8 billion tokens from cache. It recovered more budget than most AI-attributed layoff rounds save.
Route work to the right-sized model
The next lever is routing work to the appropriate model. Providers’ own price lists show flagship models costing five times their smaller siblings per token. Yet plenty of production workloads send routine classification and summarization to the most expensive tier by default. Batch processing adds a further 50% discount for anything that doesn’t need a real-time answer.
Retrieval-augmented generation attacks the problem from another angle by sending the model only the relevant slice of a knowledge base rather than the whole thing. Prompt compression trims the redundant examples that inflate every call. Open-weight models reduce costs further still, handling routine workloads at a fraction of frontier API prices for teams willing to manage the infrastructure.
These measures are simply the AI equivalent of turning off the lights in empty rooms. Uber’s $1,500 monthly cap per engineer — imposed after the April overrun — is early evidence that spending discipline arrives eventually. The companies getting ahead are simply choosing it before the budget forces it.
The other half of the fix is human
Optimizing the token bill only matters if the savings go somewhere productive. The strongest evidence points at people. Poitevin’s research found the organizations that improved ROI were those using AI to amplify their workforce rather than replace it.
Klarna ran the controlled experiment on everyone’s behalf. It replaced roughly 700 customer service roles with an OpenAI-powered assistant — and then watched customer satisfaction fall. Chief Executive Sebastian Siemiatkowski told Bloomberg what few executives admit aloud: “The result was lower quality, and that’s not sustainable.”
The fintech now runs a blended model, with AI absorbing routine volume while rehired humans handle everything requiring judgment. Gartner expects the pattern to spread, predicting that by 2027 half the companies that cut customer service staff for AI will rehire them.
The junior engineer problem
There’s one workforce investment the optimization logic makes urgent rather than optional. Stanford University’s Institute for Human-Centered AI found employment for software developers aged 22 to 25 fell nearly 20% from 2024 levels even as older cohorts grew. That means companies are removing the training ground for the senior engineers they’ll need directing all these systems in five years.
A business that has just engineered 60% off its token bill has the budget room to keep hiring at the bottom rung. Whether it does is a leadership decision, not a financial one.
Huang’s provocation will keep echoing through earnings calls, and the capex numbers will keep climbing. The companies that come out ahead won’t be the ones that spent the most on tokens or cut the most people to afford them. They’ll be the ones that noticed the token budget was the flexible line all along, squeezed it with engineering rather than headcount, and spent the difference on the people who make the tokens worth anything.
