2.8 trillion parameters, one big trade-off
On July 16, Moonshot AI dropped the Kimi K3 open-weight model, and the internet did what it always does with big numbers: fixated on the count. At 2.8 trillion parameters, K3 is the largest open-weight model ever released — jumping from Moonshot’s previous 1T-class flagship and clearing DeepSeek’s 1.6T V4 Pro by a wide margin. It’s a 3T-class model, a tier no openly available model had entered before.
But fixating on the parameter count misses the point. Moonshot’s own technical blog suggests something more specific: K3 doesn’t avoid the compute constraint so much as relocate it, trading compute for memory at almost every layer of the design. That trade is worth understanding, because compute and memory are not interchangeable constraints, and they are not equally available to a Chinese lab.
Why the Kimi K3 open-weight model is a memory problem
Two different things determine what it costs to run a large model. One is how much calculation the machine does to produce each word. The other is how much of the model has to be held ready and instantly reachable the entire time it is working. The first is compute. The second is memory. Chip export controls have squeezed China hard at the first, and Moonshot’s design reads as a sustained attempt to spend less of it.
The main move is a technique called mixture-of-experts. Rather than run the whole model for every word, K3 splits itself into 896 specialised sections and calls on just 16 of them at a time — about 1.8% of the total. The calculation per word drops sharply. The memory bill does not move at all, because all 2.8 trillion parameters still have to sit loaded and ready in case they are the ones called next.
So Moonshot went after that bill directly. It trained K3 to work at four bits of precision per parameter instead of the usual sixteen, a method known as quantisation-aware training, which the company applied from the fine-tuning stage onward and says it chose “for broad hardware compatibility” — a phrase worth pausing on, since it reads as a hedge against running on silicon that is not Nvidia‘s. The savings are substantial. Independent analysis puts the model at roughly 1.4TB in that format, against the 5.6TB it would need at full precision.
Kimi Delta Attention: attacking the context window
The second change, Kimi Delta Attention, targets a different memory cost. As a model works through a very long document, it accumulates a running store of everything it has already read. At K3’s advertised limit of a million tokens — a few thousand pages — that store, not the model itself, becomes the biggest thing in memory. Moonshot says Delta Attention enables up to 6.3x faster decoding in million-token contexts.
Moonshot is unusually direct about the commercial stake here. It contributed caching code to the open-source serving project vLLM, and says this combination is what lets it price K3 competitively despite the model’s size. The company recommends running K3 across 64 or more accelerators wired together closely enough to behave as one pool.
That is the same approach behind Huawei’s CloudMatrix systems, and it points to what the real workaround is. Memory can be gathered up across a large number of individually unremarkable chips. Training-grade compute cannot be assembled the same way.
China’s memory bottleneck — and the hardware question
Whether that pooling happens on Chinese silicon is a question the blog does not answer. Moonshot’s chip-level tests ran on Nvidia H200s and on what it describes only as a “GPGPU from an alternative vendor,” which it declines to name. Other results are benchmarked on an Nvidia L20, the cut-down card sold into China under export rules. The blog does not say where the H200 hardware sits, and the US House passed a bill in January to close the offshore cloud rental loophole that had let Chinese firms reach restricted accelerators remotely.
The reason any of this matters is that memory, not processing power, is where China’s own chip industry is furthest behind. In trade talks in August 2025, Beijing asked for relief on high-bandwidth memory restrictions rather than on lithography tools or TSMC access — a fair signal of what officials think is actually binding. Domestic output of that memory is projected at around two million stacks this year, enough for roughly 250,000 to 300,000 Huawei Ascend 910C-class chips, while SMIC has wafer capacity for more than a million.
Can enterprises actually deploy the Kimi K3 open-weight model?
For businesses in this region, the practical question is not whether K3 tops a leaderboard. It is whether an open-weight model at this size is deployable at all. Asian enterprises reach for open weights for three reasons — price, data sovereignty and regional-language coverage — and banks and insurers across Southeast Asia have been piloting self-hosted open models specifically so records never leave their own systems.
The weights land on July 27, and any organisation that can afford the hardware will be free to download, modify and run K3 inside its own walls. The question is how many can. Moonshot recommends serving the model across 64 or more accelerators wired together as a single pool, and the weights alone come to roughly 1.4TB in the format it ships in, before the memory needed to work through a long document.
That is a data-centre commitment, not a server-room one. For most enterprises, the practical outcome is renting dedicated capacity rather than owning it. That still keeps data in-country and under contract, which is what most regional regulators are asking for. What it does not deliver is the independence from infrastructure providers that drew many of these buyers to open weights in the first place.
Software readiness lags behind the launch
The software is not ready either. K3’s two main architectural changes are new enough that the standard open-source tools for running models do not yet support them, and Moonshot says it is working with inference partners and open-source maintainers to align technical details before release. Teams planning a self-hosted deployment should treat the launch date and the usable date as different things.
Pricing has shifted — and so has the competitive landscape
The price has also moved. K3 costs $3 per million input tokens, dropping to $0.30 when the model has recently seen that input, and $15 per million output tokens. That is well under Fable 5’s $50 for output, but far above z.ai’s GLM-5.2 at $4.40 and DeepSeek V4 at $0.87. K3 is no longer in the budget tier its predecessors occupied. It also launches with maximum reasoning effort as the only setting, with lower modes to follow, so long reasoning chains and retried steps add up quickly. Companies should budget on the cost of a completed task, not the list price.
What is claimed, and what is verified
Arena placed K3 first in its Frontend Code evaluation at 1,679 points, ahead of Fable 5, in blind developer testing, as reported by Tom’s Hardware. That result is real. It is also a benchmark in one domain.
Moonshot itself is more restrained than its headlines. The company states that K3’s overall performance still trails Claude Fable 5 and GPT 5.6 Sol, and lists three limitations: generation quality can become highly unstable if a harness fails to pass back historical thinking content, the model may make unexpected decisions on a user’s behalf when intent is ambiguous, and it shows a noticeable user-experience gap against Fable 5 and GPT 5.6 Sol. Its own footnotes also disclose that Fable 5 hit fallbacks on 35% of the tasks in Moonshot’s SWE Marathon evaluation, which may have affected that model’s measured score. Everything else remains a first-party claim. No published K3 number can be independently verified until the weights are public.
Bank of America analysts led by Alex Liu said in a note that K3 shows large-scale pre-training combined with architectural work can still deliver step-change gains for flagship Chinese models despite compute constraints — which is the sober version of the argument, and closer to what the blog supports.
The direction of travel is not in dispute. Open-weight models handled 29% of all tokens routed through Vercel’s production gateway in June, up from about a ninth of volume in April, while accounting for under 4% of spending. July 27 is when we find out how much of K3 belongs in that column.
For more on how open-weight models are reshaping enterprise AI strategy, read our analysis of open-source AI trends in Asia and the impact of US chip export controls on Chinese AI labs.
Social Media4 months ago
CyberSecurity4 months ago
Infosecurity4 months ago
CyberSecurity4 months ago
CyberSecurity4 months ago