The Open-Weight Gap Is Four Points Wide and 1.56 Terabytes Deep: What It Costs to Run the Best Open Models Yourself

Four points behind, and no longer catching up

Two days ago, on 27 July 2026, Moonshot AI pushed the weights for Kimi K3 to Hugging Face. It is a mixture-of-experts model with 2.8 trillion total parameters and 104 billion activated per token [7], a one-million-token context window [11], and an Artificial Analysis Intelligence Index score of 57 — the highest of any open-weights model on an index that has evaluated 170 models, 95 of them open-weights [5]. The leader of that index, Claude Opus 5 at max effort, scores 61 [4].

Four points. That is the number the debate now arrives with, and on its own terms it is real. A year before April 2026, the best open-weights model was DeepSeek V3 0324 at 22, roughly 13 points below the leading proprietary model, Claude 3.7 Sonnet with reasoning, at 35 [6]. Whatever else is true, that is a collapse in distance.

The surprise is the direction. Epoch AI's Capabilities Index folds many separate benchmarks into a single scale, so a gap can be quoted in points or in months [2]. On that scale, Epoch analysed the window from 1 January to 28 May 2026 and found that the most capable open-weight models lagged frontier closed models by an average of four months — an average ECI gap of 8 points, with a 90% confidence interval of 7 to 11 units [1]. Its previous analysis, published in October 2025, had found an average lag of three months across January 2023 to October 2025 [1]. The gap did not keep shrinking. If anything it widened slightly.

Epoch is careful about what the four-month figure is. It comes from bootstrap sampling, and treats an open-weight model as having caught up if it outperforms a previous state-of-the-art model in at least 5% of paired bootstrap samples [1]. Demand more — require the open model's ECI point estimate to be strictly higher than the closed model it is chasing — and the estimate grows to six months [1]. Epoch also names two reasons its own number may understate the gap: open-weight models tend to perform worse on private benchmarks, and leading closed labs do not always release their most capable models [1].

"Open weight" does most of the rhetorical work in this argument and very little of the technical work. Open weights means the trained parameters are downloadable. It does not mean the training code, the training dataset, or a licence that an open-source body would recognise; the Open Source Initiative's position is that open weights alone fall short of the freedoms — use, study, modify, share — that open source requires, because a model's parameters are not the preferred form in which someone would make modifications [30]. Everything that makes an open model hard to deploy — the licence, the missing training recipe, the verification a downloader must do themselves — lives in the space between the two phrases.

And it points at the thing the four-point figure hides. Every gap number in this debate is a rate with a unit sitting in the denominator, and the unit decides the width. Four points is per leaderboard entry. There is also per held-out benchmark, per completed task, per dollar of production spend, per rack of accelerators, and per hour of safety verification. The gap has a different width in each of them.

The gap is narrowest on the benchmarks that were public first

In May 2026, the US Center for AI Standards and Innovation — CAISI, at NIST — published an evaluation of DeepSeek V4 Pro against frontier US models. Its finding was double-edged in a specific and useful way. On the benchmarks DeepSeek selected and reported in its own technical report, the model looked competitive with frontier US systems. On benchmarks CAISI had committed to in advance and DeepSeek had not featured, it did not: ARC-AGI-2 semi-private, the held-out software engineering evaluation PortBench, and the cyber benchmark CTF-Archive-Diamond all showed worse performance [12].

The margins are wide.

Aggregating, CAISI put DeepSeek V4 Pro's capabilities about eight months behind the frontier [12] — twice Epoch's leaderboard-derived estimate, on a suite the model's developers could not have optimised toward.

CAISI anticipated the objection that it had misconfigured the model: it served DeepSeek V4 from cloud-based H200 and B200 GPUs and reproduced the developer's own self-reported GPQA-Diamond result before running anything else, specifically to rule out an inference or configuration error [12]. That does not make the eight-month figure a law. It is one evaluator's aggregate over one held-out suite, and IRT-estimated Elo is a modelled quantity, not a measured one. But it does mean the difference between four months and eight months is unlikely to be an artefact of who ran the model.

Epoch reaches the same place from the other side, and with appropriate hedging: it notes evidence that open-weight models tend to perform worse on private benchmarks than on public ones, plausibly because they hillclimb public benchmarks more aggressively [1]. Plausibly is Epoch's word, and it is the right strength. Nobody has isolated training-set contamination as the mechanism; what has been observed is a consistent gap between public and private results.

Not on every private benchmark, though. AA-Briefcase is Artificial Analysis's proprietary long-horizon knowledge-work evaluation, run on a fully private dataset of realistic tasks built around thousands of complex input files, scored on correctness, analytical quality and presentation quality. On that benchmark, in the results Artificial Analysis posted on X at the time of the release, Kimi K3 placed second overall, behind only Claude Fable 5 [17]. Its rubric scoring and analytical quality came close to Fable 5's [3]. Fable 5's Elo on that benchmark is 1574; Claude Opus 5 has since taken the top of it at 1720 [15], so K3 now sits third rather than second. On the separate GDPval-AA v2 agentic evaluation, K3 reaches an Elo of 1668, surpassing GLM-5.2's 1514, GPT-5.5's 1494 and Claude Opus 4.8's 1600, while remaining behind Claude Fable 5's 1760 [3]. A private benchmark did not automatically expose the open model. The private-benchmark penalty in the CAISI results is real but concentrated: it appears in reasoning under novelty and in adversarial security work, not uniformly.

So the size of the gap depends on which test you run. It also depends on what you count.

Per token, or per finished task

Per token, open weights are not close to the frontier. They are far below it. DeepSeek V4 Flash costs $0.14 per million input tokens and $0.28 per million output tokens — on input, 18 times cheaper than GPT-5.4 at $2.50 and 36 times cheaper than Claude Opus 4.7 at $5.00, on one cost-tracker's comparison [18]. Claude Opus 5's tokens cost $5 and $25 per million, with cache hits at $0.50 [19]. That is the comparison that has carried the open-weight argument for two years, and as arithmetic it is unimpeachable.

It is also the wrong denominator, because nobody buys tokens. They buy finished work.

Artificial Analysis reports cost per Intelligence Index task alongside the index score, and on that measure the advantage shrinks to something ordinary. Kimi K3 costs an average of $0.94 per Intelligence Index task — similar to GPT-5.6 Sol at $1.04, and about half the price of Claude Opus 4.8 at $1.80 [3]. Claude Opus 5 at max averages $2.03 [19]. A price advantage that reads as tens of times per token reads as roughly half on a bounded task, in part because Moonshot's own pricing for K3 sits far above its pricing for K2: K3's output tokens are $15 per million against K2.6's $4 [3].

Now price the same models on long-horizon agentic work, where the task is not one prompt but a session. On AA-Briefcase, Kimi K3 averaged $10.57 and 56.4 minutes per task [9]. Claude Opus 5 at max effort averaged $17.79 per task and 36.2 minutes [15]. The closed model still costs more — but by well under double, while finishing in roughly two-thirds of the time. Push one step further and the advantage disappears: Claude Opus 5 at high effort costs $10.41 per task, level with K3 — and at that price it outperforms Claude Fable 5 by 32 Elo [15].

The mechanism is turns and output tokens. Artificial Analysis, posting its AA-Briefcase results on X, reported that K3 averages 83 turns per task against 67 for Claude Fable 5 and 50 for GPT-5.6 Sol at max, and that its AA-Briefcase cost is roughly ten times its own predecessor's [17]. A model that reaches a comparable answer by taking more steps and emitting more tokens will erase a per-token discount, because the discount is per token and the bill is per session.

The control matters here, because turn growth is not an open-weight defect. Claude Sonnet 5, a closed model with cheaper tokens than Opus 4.8, costs $2.29 per Intelligence Index task without promotional pricing — about 15% more than Opus 4.8 — and Artificial Analysis attributes that entirely to increased token usage, noting that at max effort Sonnet 5 used around 40% more output tokens per Intelligence Index task than Sonnet 4.6 and roughly three times the agentic turns on AA-Briefcase and GDPval-AA [16]. Per-token price is a poor proxy for cost of work in general, and it misleads in both directions. The open-weight case is not uniquely guilty of it; it is merely the case that leans on it hardest.

What survives is a narrower and more useful claim than the one usually made. Per token, the open-weight advantage is a factor of tens. Per bounded evaluation task, it is a factor of about two. Per long-horizon agentic session, it is gone: against Claude Opus 5 at high effort, the open model is no cheaper.

Production routing already split on that seam

The market has already priced this, and the split is visible in gateway telemetry.

Vercel's AI Gateway Production Index for July 2026 reports that in June, open-weight models ran 29% of gateway tokens on just under 4% of spend, up from 11% of tokens in April [20]. Anthropic took 61% of gateway spend on 32% of tokens, and captured 72% or more of spend in coding agents, back-office agents and app generation [20]. Nearly a third of the tokens for one twenty-fifth of the dollars, in Vercel's own summary [20].

OpenRouter's data agrees on the shape. An empirical study of 100 trillion tokens routed through OpenRouter found open-weight models growing steadily to roughly one-third of token usage by late 2025 [21].

And Menlo Ventures' enterprise survey says the opposite. In its 2025 State of Generative AI in the Enterprise, open-source model share of enterprise workloads fell from 19% in 2024 to 11% in 2025, with Llama still the most widely adopted open-weight model but stagnating after Llama 4 [22]. Chinese open-source models collectively account for just 1% of enterprise LLM spend [22].

Both readings are correct, and the contradiction is the point. Tokens routed, dollars committed and workloads owned are three different denominators. Open weights win the first decisively, lose the second by a factor of roughly twenty-five, and are losing ground on the third. Anyone quoting one of these numbers as the standings is quoting a denominator and calling it a verdict.

The seam is also live rather than settled, and the most recent movement runs the other way. Guillermo Rauch, Vercel's chief executive, posted a thread on X drawing on the same gateway data: Moonshot's share of gateway spend climbed steeply over a single week in July, and the combined spend share of closed models fell to an all-time low within days of hitting an all-time high in late June [23]. That is a single platform over a single week, and gateway telemetry captures the traffic that flows through one gateway, not the market.

Still, the standing pattern is clear enough to read. Running 29% of tokens on under 4% of spend describes organisations buying open-weight inference from someone else, cheaply, for work where an error is affordable. It does not describe organisations running open weights themselves. That distinction is where the real cost lives.

Owning the weights buys provider choice, not deployment

Kimi K3's Hugging Face repository occupies 1.56 TB across 96 safetensors shards, plus configuration files and custom code for processing text and images [9]. The weights themselves are approximately 1.4 TB even at MXFP4 [26] — a compression Moonshot achieved by applying quantisation-aware training from the supervised fine-tuning stage onward, using MXFP4 weights with MXFP8 activations for broad hardware compatibility [7], rather than quantising a finished model after the fact.

For deployment, Moonshot's recommendation in its own technical write-up is a supernode configuration of 64 or more accelerators, on the reasoning that inference efficiency benefits from larger high-bandwidth communication domains [8]. Secondary reports put the requirement considerably lower: one account of the release describes the hardware needed as eight B300 GPUs, sixteen H200s, or thirty-two 80 GB H100s [9]. The two figures measure different quantities: what will run the model at all, against what Moonshot says will run it efficiently. Northflank's self-hosting analysis, working from the same public material, reports the same recommendation — a supernode configuration with 64 or more accelerators [27]. So the floor sits somewhere between eight and sixty-four data-centre accelerators, depending on whose number you take and what you are optimising for, and only the upper figure comes from the vendor. Either way, this is a data-centre commitment, not a workstation.

The economics of taking that commitment on are dominated by a variable most estimates never measure. A 2026 methodology paper on LLM infrastructure cost makes the case directly: utilisation is almost always entered by the user, baked in as a preset, or silently assumed at 100%, never measured against the operator's actual load [24]. On identical H100 hardware, the authors find effective cost spans $0.21 to $15.25 per million output tokens — an underutilisation penalty of 2.5 to 24 times across low-to-moderate enterprise loads of 1 to 10 requests per second, and up to 36.3 times near idle [24]. The penalty is driven by one operator-controlled variable, the offered request rate, which sets in-flight concurrency. A rented GPU costs the same whether it is busy or not, so a self-hosting estimate that assumes full utilisation understates the true cost by exactly the reciprocal of the utilisation actually achieved.

That penalty is what break-even estimates leave out. One 2026 cost comparison puts the crossover for self-hosting open models against closed-model APIs at roughly 2 to 5 million tokens per day, and against open-model API providers at 50 million or more [25] — the second threshold being the relevant one, since a team choosing an open model is usually choosing between hosting it and buying it from a provider that already does. Those are secondary estimates built on assumed load, and the utilisation result is what tells you how far they can move. For frontier mixture-of-experts models specifically, hosted APIs are frequently cheaper outright, because providers batch many tenants onto the same accelerators and reach utilisation a single deployer cannot.

Then there is the licence, and here "open weight" stops being a binary and becomes a spectrum whose frontier end is the restricted end.

Kimi K3 did not ship under a permissive licence. It shipped under the bespoke Kimi K3 License, which requires any licensee operating a Model as a Service business with aggregate revenue above 20 million US dollars over any consecutive twelve months to enter a separate agreement with Moonshot before commercial use, and requires products with more than 100 million monthly active users or more than 20 million US dollars in monthly revenue to display "Kimi K3" prominently in their user interface [10]. The release was widely described as being under a Modified MIT licence — a claim that circulated in community discussion on Reddit within hours [35] — and it is not; Hugging Face categorises the licence as other, not as an SPDX-recognised MIT identifier [26]. A licence that most readers believed was MIT is one that gates the exact commercial use a serious deployer would have in mind.

Llama has gated at 700 million monthly active users since Llama 3, above which a licensee must request a separate licence from Meta, granted at Meta's sole discretion [28]. For Llama 4's multimodal models, Meta's acceptable use policy withholds the rights granted under Section 1(a) of the community licence from individuals domiciled in the European Union and companies with a principal place of business there [29].

The Open Source Initiative's position is that none of these are open source in the first place, and not because of the thresholds. Open weights provide the final parameters of a trained model without the training code, the training dataset where legally possible, or comprehensive data transparency — which forecloses reproducibility, auditability and the collaborative modification that the four freedoms assume [30]. There is a genuine counterweight: Google moved Gemma 4 to the Apache 2.0 licence, abandoning the custom terms of earlier Gemma releases [31]. But the pattern at the capability frontier runs the other way, and Gemma 4 is not at the capability frontier.

Regulation compounds the pattern rather than relieving it. The EU AI Act exempts providers of general-purpose AI models released under a free and open-source licence from parts of Article 53's documentation obligations — but the European Commission's own Article 53 guidance states plainly that the exemption does not apply to models classified as general-purpose AI models with systemic risk [32]. Systemic risk is presumed when cumulative training compute exceeds 10^25 floating-point operations, and providers of such models owe the additional Article 55 duties: model evaluation, risk mitigation, and incident reporting [33]. So the carve-out falls away exactly where a model carries systemic risk, and what its provider owes instead is evaluation and reporting work.

The last denominator is assurance

Those duties fall on the provider. The deployer has a version of the same bill, and nothing in the download covers it: before an open model runs on your infrastructure, someone on your side has to establish how it behaves. That is the last denominator, and plausibly the cost sitting behind the spend split.

CAISI's September 2025 evaluation of DeepSeek tested three DeepSeek models — R1, R1-0528 and V3.1 — against four US reference models on 19 benchmarks [13]. On security it found a difference of kind rather than degree. Agents built on DeepSeek's most secure model, R1-0528, were on average 12 times likelier than the evaluated US frontier models to follow malicious instructions designed to derail them from user tasks [13]. That model complied with 94% of overtly malicious requests using common jailbreaking techniques, against 8% of such requests for the US reference models [14].

Two limits bound that finding.

The first is scope. That evaluation is from September 2025 and covers R1, R1-0528 and V3.1 specifically [13]. It is evidence about three DeepSeek models at one point in time, not a general law about open weights, and treating it as one would be exactly the overreach the numbers do not support.

The second is that the newest comparable data point runs the other way. Trade coverage of a joint evaluation by CAISI and the UK AI Security Institute reports that Kimi K3 scored 32% on the 41-task ExploitBench cyber evaluation and achieved arbitrary code execution on none of the tasks, while the most capable US models averaged 20 successful cases [9]. On that one benchmark the open model did not come out the more dangerous of the two.

There is also an axis that gets filed as an open-versus-closed failing and is nothing of the kind. Kimi K3's hallucination rate hit 51% [34], which reads badly until it is set beside the closed frontier: one data-labeling vendor's analysis puts Claude Fable 5 at 54.9% on unfamiliar questions, higher than K3 [34] — figures from one elicitation protocol, not a ranking — and Claude Opus 5's hallucination rate is 50%, up 14 points from Opus 4.8 because it answers more often when uncertain [19]. Anthropic's newest model got worse on this measure while improving on the capability evaluations around it. That pattern points at benchmark incentives rather than at openness: a binary-graded evaluation penalises an abstention exactly as much as a wrong answer, so a model tuned to answer more often scores worse on hallucination while scoring better on accuracy. Open and closed models are moving the same way here.

What survives all three observations is not a ranking but a cost. Assurance is work. It does not ship with the weights, it is not covered by a permissive licence, and it has to be redone for each model a deployer adopts. It is the most plausible explanation on the table for why 61% of gateway spend sits with a single closed vendor while 29% of tokens sit with open models [20] — though it is an explanation, not a demonstrated cause. Gateway telemetry records what was routed where, and nothing here isolates verification cost as the reason.

So which gap are you paying?

The standings question has no scalar answer. Which gap you are paying is a fact about your workload, not about the models. Short bounded calls where an error is cheap: the per-token gap is the one that matters, and open weights win it by a wide margin. Long agentic sessions with a deliverable at the end: the per-session gap is the one that matters, and it is close enough that effort settings decide it. Regulated or air-gapped deployment: the gap is a rack and a legal review, and it is not close.

One thing here is general. The direction of travel stopped being about capability at roughly the moment the capability gap stopped narrowing — across Epoch's window, which opens in January 2026, the lag averages four months [1]. Since then the question has not been whether an open model can do the work. It has been who absorbs the cost of turning a downloadable file of weights into a running system — and nothing here shows that cost falling the way the benchmark gap did.

One question none of this answers is worth naming, because it is the strongest classic argument for open weights: fine-tuning. Whether owning weights you can adapt beats renting a model you cannot is not something the evidence here measures. The one relevant observation is that post-training Kimi K3 is extremely difficult, because loading the weights takes an entire GPU node before any training begins [36] — suggestive that the customisation advantage inverts at frontier scale, but a single observation is not a finding.