Which LLM coding subscription is actually fast?

A methodology change and a few board moves this week.

We now measure time-to-first-token. Every speed benchmark streams and separates TTFT from total time; we report a generation rate (tok/s excluding the prefill/queue wait) alongside the end-to-end figure.

The DeepSeek 5,000+ tok/s mystery is solved — it was a measurement artifact. The old single-elapsed timing folded queue and prefill into the rate. With TTFT separated, DeepSeek lands at ~60-75 tok/s end-to-end. That closes the “more next week” note from the index.

What TTFT exposes. End-to-end tok/s was hiding large prefill waits: CrofAI's kimi-k2.5-lightning generates at ~6,500 tok/s once it starts but carries an 11.7s TTFT; the Synthetic and opencode-go GLMs lose 7-10s to prefill. Cerebras is the genuine speed leader — llama3.1-8b at 1,479 tok/s and gpt-oss-120b at 1,372, both with 0.2-0.3s TTFT.

CrofAI is off the board — a serving outage. For the last few hours CrofAI has been returning HTTP 200 with empty bodies across much of its catalogue (the -precision/-pro/-flash variants and the newer, bigger models: deepseek-v4-pro-precision 71% empty, gemma-4-31b-it 55%, glm-5.1-precision 45%). It also hit the model we use as our cost-optimized judge, so this week we re-judged the architecture suite against our reference judge, claude-opus-4-7. CrofAI comes back when serving stabilises — and yes, that moves it out of the pick too (it was last week's pick; when something breaks, we say so).

OpenAI is retired from the board — our call. We brought ChatGPT in for a few weeks to evaluate it and aren't convinced it earns a permanent column, so we're dropping it (the larger gpt-5.x and -codex models also need a Pro subscription we no longer keep). Want a provider covered? Sponsor it. Lend us an API key with modest quota — single-digit millions of tokens a month is enough — and we'll bench it under the same disclosure rules as everyone else. API-credit donations welcome: millaguie [at] gmail [dot] com.

Also out: Synthetic finished its retirement pass (DeepSeek V3.x, Llama 3/4, Qwen2.5-Coder now return 404); alibaba/qwen3.7-max didn't respond (errors across the run); Copilot stays free-tier only.

Quality (architect, judged by opus-4-7). Top of the board: claude-sonnet-4-5 (0.976) and claude-opus-4-7 (0.974), then kimi-k2.5 on Alibaba (0.967), Kimi-K2.6 on Synthetic (0.966) and claude-sonnet-4-6 (0.966). Note: switching the judge from kimi-k2.6-precision to opus-4-7 recalibrates absolute scores, so this week's architect numbers are not directly comparable to the W18-W20 trend line.

Five things landed this week.

opencode-go debuts — and falls 40% behind on its flagship. The opencode team shipped their own pay-as-you-go gateway (opencode.ai/zen/go/v1), a curated open-source catalogue of twelve models (DeepSeek V4, GLM 5.x, Kimi K2.5/K2.6, MiMo V2.5, MiniMax M2.5/M2.7, Qwen 3.5/3.6). Convenient — one key for what opencode itself uses — but on quality benches it trails the direct providers it routes to. Our drift detector flagged deepseek-v4-pro on opencode-go at -40 % vs DeepSeek direct (critical, z=-2.20), kimi-k2.6 at -20 % vs CrofAI, and minimax-m2.7 at -9 % vs MiniMax direct. The lighter models fare better (minimax-m2.5 placed #7 in the combined ranking), but the routing markup hits the heavy reasoners disproportionately. Useful as a convenience layer; not a replacement for the upstreams.

OpenAI is off the bench this week. Our ChatGPT Pro subscription — the one we measured the gpt-5.x family against — lapsed before W20 started, and several of the larger models (gpt-5.5, gpt-5.5-fast, all -codex variants) require Pro to access. Rather than publish a half-measured OpenAI column, we pulled it entirely for the week. It comes back when we either renew or switch to a different auth flow that survives the Pro lapse. Historical W18-W19 OpenAI data on the dashboard stays.

GitHub Copilot, also off — but for editorial reasons. Microsoft's recent platform direction has made it less interesting to bench as a coding-agent option (model availability per plan, rate-limit changes, and the new opt-outs all moving in directions that erode what made Copilot worth the column). The auth is still wired and we keep the historical numbers, but we're not running Copilot in the weekly bench until that picture stabilises.

Catalogue churn. CrofAI added four models this week (deepseek-v4-flash, deepseek-v4-pro-precision, mimo-v2.5-pro, mimo-v2.5-pro-precision) and retired qwen3.5-9b-chat. The Xiaomi MiMo line lands well: mimo-v2.5-pro-precision sits #11 in the architect rankings at 0.972, holding its own against Qwen and GLM at similar tok/s. Synthetic, meanwhile, ran a big retirement pass — fourteen models now return HTTP 404 "no longer supported", including the Llama 3.x/4 family, DeepSeek V3 through V3.2, Qwen2.5-Coder, MiniMax-M2/M2.1, the Kimi-K2-Instruct/Thinking pair and GLM-4.6. Their catalogue is consolidating around current SKUs only.

baidu/cobuddy:free is the genuine surprise. Topped the architect rankings outright (0.989), placed mid-table in developer (0.884), and clocked a 499 tok/s median throughput — all with no per-token cost on OpenRouter's free tier. We've eyeballed a handful of its responses and they hold up; this isn't the usual :free tradeoff of "works until it doesn't." If you need a second lane behind your paid daily driver, this is currently the strongest free option on the board.

How opencode-go and Alibaba Cloud got onto the bench. Both providers are on the leaderboard because somebody at — or with — the provider sent us an API key with enough quota to run the standard weekly suite. We will do the same for any provider we're not currently covering. The deal stays the same as for everyone else already on the board: we wire the auth, we publish the numbers (the unflattering ones included), and we annotate the methodology so readers can reproduce. If you can spot us a key — single-digit millions of tokens per month is enough — write to millaguie [at] gmail [dot] com or open an issue on the repo.

Heads-up on Alibaba Cloud. Starting next month, Alibaba's updated Terms of Service will rule out the kind of access this benchmark relies on. We expect to lose the Alibaba column from the next monthly cycle onward — keys revoked, numbers frozen. The historical W18-W20 Alibaba data on the dashboard stays so the trend lines remain intact, but the weekly leaderboard will no longer include them once the new ToS takes effect. If Alibaba shipped a compliant path for third-party benches before then, we'd switch to it; right now there isn't one.

Three things landed this week.

OpenRouter as a new front door. 187 models in their catalogue, 25 of them :free text models. With $10+ in credit you cross into the higher daily quota tier (~1000 free-model requests/day), making them viable for both speed and quality benches. Two surprises in the speed pass: nvidia/nemotron-3-nano-30b-a3b:free clocked 4177 tok/s and poolside/laguna-xs.2:free hit 3746 tok/s — both in Cerebras territory, unusual for OpenRouter-routed inference.

Catalogue refresh. Most providers shipped new SKUs since W18: Claude added Sonnet 4.5; Copilot opened up the gpt-5.x family plus grok-code-fast-1; Alibaba's Qwen3.6 lineup is in; Cerebras now serves zai-glm-4.7; CrofAI gained five Kimi and qwen3.5 variants; OpenAI's gpt-5.5 family and several gpt-5.4 variants are tested for the first time.

Alibaba opened a second door — the Token Plan (Team Edition). After months of the legacy Coding Plan being effectively unobtainable, Alibaba shipped a parallel Token Plan that you can actually buy. Three tiers per seat: Standard $30/mo (25k Credits), Pro $100/mo (100k Credits, their recommended tier), Max $200/mo (250k Credits). Floor is cheaper than the legacy $50 Pro, but the closest equivalent is now $100 — a 2× hike at the volume most users actually want.

Credits vs requests — what really changed. The legacy Coding Plan billed by request count (one HTTP call to the endpoint). A single user query in an agent like opencode or Claude Code typically expands into 5-30 requests internally — planning, tool calls, re-prompts — so 90k requests/mo at $50 actually meant a few thousand user-visible queries. The Token Plan bills by Credits derived from input + cached + output tokens, modulated by model, thinking mode and tool calls. Alibaba doesn't publish the credit-to-token ratio, so budgeting precisely is impossible without measuring your own workload first. Practical effect: request-based subsidised thinking-mode and long-output models (they cost the same as a one-shot reply); credit-based makes you pay for those tokens proportionally. Short verbose-tool-using agents are better off on Credits; long-context, deep-reasoning, terse-output workloads were better off on the old request quota — which you can't buy anymore.

Catch: the Token Plan ToS restrict it to interactive use with compatible AI coding/agent tools only. What the FAQ explicitly forbids: automated scripts, application backends, benchmarking and research scripts — violations may trigger API key revocation. What stays allowed: long agentic sessions inside a compatible tool (Claude Code, opencode, Qwen Code, etc.) with a human in the loop, even if the agent itself runs autonomously for hours. The line is "is there a person driving a compatible tool?", not session duration. Our weekly bench against Alibaba runs against a legacy Coding Plan subscription where this clause didn't exist; new subscribers on the Token Plan should keep usage inside one of those compatible agents.

A note on :free quotas. OpenRouter's free tier resets daily at midnight UTC, but the per-model bottleneck is usually the upstream provider, not OpenRouter itself. Several Meta-llama and Mistral-derived variants returned HTTP 429s all week and never recovered across 11 retry attempts — those are upstream pool limits, not your account's. Cross-provider drift comparisons (openai/gpt-oss-120b OpenRouter vs Synthetic; z-ai/glm-4.5-air OpenRouter vs zai) are now possible and arguably the real new value here.