Deep dive: open vs closed models

Open-weight vs closed models: what's real

My view as of October 2026: below the frontier, an open-weight model is good enough for most chat, summarization and extraction work, at a half to a sixth of the price, and a gateway makes moving to it a configuration change. At the frontier, especially agentic coding, the gap reopens with each closed release and most enterprise money stays closed. The real switching cost is evals, prompt rework, checking each host and reading the contract, more than changing the API call.

How big the gap is

• On the main composite index: in Artificial Analysis's April 30, 2026 snapshot, the best open-weight models (Moonshot's Kimi K2.6 and Xiaomi's MiMo V2.5 Pro) scored 54 against 60 for OpenAI's GPT-5.5 at its highest effort, a 6-point gap, and open models held 9 of the 13 spots on its frontier of intelligence against price. By October, on a newer version of the index, Xiaomi's MiMo-V2.6-Pro scored 46 against 58 for Anthropic's Opus 5.5 and 53 for the best of OpenAI and Google, about 12 points. Index versions rescale, so compare only within one.
• On hard tasks the gap is wider: 43-46% for the best open models against 61% for closed ones on Artificial Analysis's hard terminal-coding tasks in April.
• Who makes them: the top six open-weight models in October were all Chinese. The US ones are OpenAI's gpt-oss, Google's Gemma and Nvidia's Nemotron, and Meta has moved its frontier work to closed models.

What open costs

The price gap is large at the small and mid end: gpt-oss-120b at $0.15 / $0.60 and DeepSeek V4.1 Flash at $0.30 / $1.20 per million tokens, against $2 / $10 for a closed mid-tier model. It narrows at the top, where Kimi K3 lists at $3 / $15, close to Opus 5.5's $4 / $20. Serving a big open model yourself takes a full node of eight B200s per replica.

Same model, different host

• Quality varies: on a 2025 maths test run 32 times, gpt-oss-120b scored 93.3% at seven hosts, 86.7% at Groq, 83.3% at Amazon, 80% at Azure and 36.7% at one small host. Hosts differ in precision, chat templates and serving bugs.
• Speed varies more: about 37 times between the slowest and fastest gpt-oss host in October 2026, and up to 8.9 times on price.

So "route to the cheapest host of the same model" needs a per-host eval, and a failover to the same model on another host can still change your answers.

Where the money actually goes

Usage share and money share point different ways. On OpenRouter, open-weight models were about a third of tokens by late 2025. Datadog found more than 70% of organizations using three or more models. But a 10% price cut moved usage by only 0.5-0.7%, and four in ten users of one 2025 Claude model were still on it five months later. In Menlo Ventures' late-2025 survey (Menlo is an investor in Anthropic), Anthropic, OpenAI and Google took 88% of enterprise LLM spend between them, and by mid-2025 the open-source share of enterprise workloads had fallen from 19% to 13%. Quality and habit decide the vendor; price decides the long tail.

Jurisdiction and contracts

Open weights move the contract risk to you. Self-hosted, there's no retirement clock and no indemnity; at a provider, the model maker isn't your counterparty, and most IP indemnities name only the vendor's own models. Chinese labs aren't Code of Practice signatories and US restrictions were under discussion in 2026, while sovereign regions tend to launch with open models. For a team selling to US government-adjacent buyers I'd keep Chinese-model routes behind a switch I can turn off per customer.

Questions to ask a provider

1. Which precision and serving engine do you run this model on, and do you pass the model maker's own verification test?
2. What are your median and 95th-percentile time to the first token and output speed on our prompt sizes, measured by someone else?
3. What notice do you give before dropping a model from serverless, and what happens to our dedicated endpoints?
4. Is our data retained or used, by you or by any sub-processor, and can you give zero retention in writing?
5. Which of your models, if any, come with an IP indemnity?