Open-weight vs closed (proprietary) models.
The published gap between open-weight and closed models is a lower bound, and the measurements say so: Epoch AI's capabilities index put the best open-weight models about four months behind the closed frontier through 2026, while its own summary across seventeen selected benchmarks found the gap on private benchmarks running eight to ten months against four to six on public ones — roughly twice as wide where the test set could not be optimised against. So the headline comparison tells you about measurable things, and your task is not one of them yet. The other half of the decision is sharper than the debate suggests: renting an open-weight model buys you a four-to-sixfold discount and none of the control arguments, because every one of those requires you to actually run the weights.
Step 1Get the terminology straight, then get the gap right.
Three words, and the first two are not synonyms. Open weight means the trained parameters are published — you can download them, serve them, usually fine-tune them — while the training data and full training code generally are not. Fully open source adds the code and ideally the data under an open licence, and is much rarer. Closed means the weights are never published and the model exists for you only as a hosted API. Licences vary enormously inside "open weight", from near-permissive to source-available with acceptable-use and scale conditions, and "you can download it" has never meant "you can use it however you like" — the specifics are in open-weight licences.
Now the capability question, with dates attached because every number here decays. Epoch AI's May 2026 analysis, using its Epoch Capabilities Index, found that since January 2026 the most capable open-weight models had trailed the frontier closed models by about four months, an average index gap of 8 points — roughly the distance between consecutive releases in one vendor's own line. Its earlier look, covering 2023 to late 2025, had put the lag nearer three months, so the gap has drifted slightly wider rather than closing. As of October 2026 the open side of that comparison is led by models such as Kimi K2.6, GLM-5.1 and Qwen 3.6 Plus, against a closed frontier led by GPT-5.5 Pro, Claude Opus 4.7 and Gemini 3.5 Flash.
Two caveats do more work than the headline. First, the index is a composite: it is silent about any single domain, and the domains diverge hard — the coding gap on SWE-Bench-style tests is small enough that one agent evaluation put it around 7 points and not statistically clear, while reasoning-heavy tests show a visible spread. Second, and more importantly, Epoch noted its own estimate may understate the true gap because open-weight models appear to score relatively worse on private benchmarks than on public ones; its summary across seventeen selected benchmarks put the private gap at eight to ten months against four to six public. That is a measurement about measurement, and it is the one that should change how you read every leaderboard — the full treatment is in reading benchmarks critically.
The practical reading is not "open models are secretly worse". It is that a public benchmark is a target, and anything aimed at a public target tells you less about an unseen task than its score suggests — the mechanism in benchmark contamination. Treat published open-versus-closed comparisons as a shortlisting filter and nothing more. The number that decides this is the one from your own held-out eval set, which is also the only asset in this decision that keeps its value when every model on both sides is replaced.
Step 2Renting open weights buys price. Running them buys everything else.
This is the distinction the debate usually collapses, and collapsing it is the most common expensive mistake here. The arguments on the open side of the ledger come in two separate bundles with different prices.
What you get from a hosted open-weight endpoint — a third-party provider serving Kimi, GLM or Qwen through an API — is the discount, plus the option to leave. As of October 2026, published list prices put Kimi K2.6 around $0.95 per million input tokens and $4.00 output, GLM-5.1 around $1.40 and $4.40, against roughly $5.00 and $30.00 for GPT-5.5 and $5.00 and $25.00 for Claude Opus 4.7. That is about four to five times on input and six to seven on output. The portability is real too: several independent providers serve the same weights, so your exit is a base-URL change rather than a rewrite — the genuine lock-in argument, and the main one.
What you do not get from that endpoint is any of the rest of it. Your data still leaves your boundary, so the residency argument in data residency and sovereignty does not apply. You cannot pin a version the provider retires, so the reproducibility argument does not apply. You have no idea what quantization is being served — the serving-contract problem in serving and access — so "same weights" does not mean same behaviour. Deep fine-tuning needs weights you hold, not weights someone else serves.
Those four arguments require actually running the model, which is a different business. It is capex-shaped rather than opex-shaped: accelerators cost the same whether or not they are busy, so the crossover is entirely a utilisation question, and it arrives much later than people expect once the engineering and reliability cost of serving inference well is counted honestly — the arithmetic is in self-hosted inference for agents and build vs buy.
# The three options, and what each actually delivers. price exit residency version deep break option control pinning fine-tune closed API — weak no no no hosted open weights 4-6x strong no no no self-hosted weights util. strong yes yes yes depend. # The expensive mistake: picking hosted open weights for a # residency or pinning requirement it does not satisfy.
Step 3Do the arithmetic on your own traffic before the argument.
Take a support-triage agent at two million requests a month, averaging 3,000 input and 400 output tokens per request — six billion input and 0.8 billion output tokens. At GPT-5.5's listed rates that is roughly $30,000 of input and $24,000 of output, about $54,000 a month. On hosted Kimi K2.6 at its listed rates it is roughly $5,700 and $3,200, about $8,900. Call it a $45,000 monthly difference, which is a real decision and not a rounding error.
Now price the two things that usually eat it. If the cheaper model needs a second pass on 15% of requests, or thinks longer to compensate for weaker weights, the output-side cost moves by more than the headline ratio suggests — reasoning spend is billed at the output rate and is easy to miss entirely, which is the point of accounting for reasoning tokens. And if 10% of requests have to escalate to the frontier model for quality, you are paying $8,000 of cheap traffic plus $5,400 of expensive traffic and the saving is still most of the way intact — which is why the hybrid is the normal answer rather than a compromise.
That hybrid is worth stating as structure rather than as a hedge: a cheap or local model on the high-volume path, a frontier API for the hard and rare cases, behind a router you own. It is the model routing pattern, and it also happens to be the strongest anti-lock-in position available, because the expensive dependency sits behind an interface you control instead of being wired through your application. A model small enough to run on your own hardware adds a third tier — the case made in small and local models.
Two cost terms that routinely invert this comparison. Cached input is priced separately and aggressively — around $0.16 per million for Kimi K2.6 and $0.26 for GLM-5.1 on their published rates — so a workload with a large stable prefix has a cost structure that list prices do not describe at all (prompt caching). And context pricing steps: several closed models charge a higher rate above a threshold — GPT-5.5 above roughly 272K tokens, Gemini 3.5 Flash above 200K — while Claude Opus 4.7 is listed with a one-million-token window and no long-context surcharge. A long-context agent and a short-context one should not reach the same conclusion.
Step 4A decision rule that survives the next release cycle.
Answer these in order and the choice usually makes itself, without anyone having to predict who is ahead next quarter.
- Must the data stay inside your boundary? If yes, the only option is weights you run. Everything else on this list is moot, and the question becomes which servable-size model clears your bar — not open versus closed.
- Do you need a version frozen for longer than a vendor will keep it? Regulated model-risk regimes and long-lived evaluation baselines both want this, and only self-hosting delivers it. A closed snapshot will be retired on the provider's schedule — model deprecation and migration.
- Is your quality bar at the top of the frontier? Measure it on your own set, not on a leaderboard, and remember the private-benchmark finding: the open option will look relatively better on a public test than on yours. If the honest answer is that you need the frontier, pay for it on the path that needs it and only that path.
- Is sustained volume high and the task narrow? This is where open weights win on economics, and where a fine-tuned small model often beats both — RL fine-tuning open weights, with the serving-cost reductions in distillation and quantization.
- Is the real worry lock-in? Then build the router first and choose the model second. A hosted open-weight model behind an abstraction is a strong exit; a frontier model behind the same abstraction is a weaker but still real one. A frontier model called directly from forty places in your codebase is the thing you were actually afraid of.
Do this instead of having the argument: build a hundred-item eval set from your own traffic, with the answers you would accept, and run your current closed model, one leading open-weight model on a hosted endpoint, and one small model you could serve yourself. Score them, and price each at your real volume including cached input. You will typically find the gap on your task is either much smaller or much larger than the four-month headline, and in both cases you now own the one measurement that does not expire when the next model ships — the asset argued for in choosing a model. "Open" is not automatically cheaper, safer or more private; it is a specific discount with a specific set of control arguments attached, and only some of them come with the endpoint you were about to call.