Runs on ≤24 GB · Hugging Face ↗
Not independently scored by Artificial Analysis
A Mixture-of-Experts coder — 30B total but only ~3B active per token, so it runs faster than its size suggests while all experts must still sit in VRAM. Purpose-built for agentic / tool-use coding.
HumanEval pass@1 98% (39/40), via OpenRouter. Measured by us on this hardware, as a rough sanity check — HumanEval is a different, easier, partly-contaminated benchmark than Artificial Analysis’ composite, so it is not comparable to the coding-index column.
BigCodeBench-Hard pass@1 32% (48/148), via official protocol · llama.cpp Q4_K_M · one request at a time · a shared cluster GPU · mean of 3 runs, range 0.0 pp. The brutal counterpart to HumanEval — where HumanEval saturates near the top, BCB-Hard spreads the field, so this is the number that actually separates coding ability. 1 of 148 answers hit the token limit (effective ceiling 99%)
Our own runs on this model. Each number carries how many problems it is over and how many times we ran it — these are not comparable with each other, and none of them is on the same scale as the third-party indices on the leaderboard.
| test | score | over | conditions |
|---|---|---|---|
| BigCodeBench-Hard | 32.4% | n=148 · 3 run(s) | official protocol · llama.cpp Q4_K_M · one request at a time · a shared cluster GPU · mean of 3 runs, range 0. |
| BFCL v4 non-live | 85.4% ± 2.3 | n=1150 · 1 run(s) | BFCL v4 · Q4_K_M GGUF via llama.cpp · one request at a time · a shared cluster GPU |
85.4% ± 2.3 over 1150 problems
Q4_K_M · a shared cluster GPU · KV f16 · 1 run · spread between runs assumed at 0.25 points, not measured. This is the non-agentic half of BFCL: does the model call the right function with the right arguments. It says nothing about how it behaves over a long agentic conversation, which is a separate measurement. The score is BFCL’s own: the unweighted mean of four groups (simple, multiple, parallel, parallel multiple), where simple is itself the mean of Python, Java and JavaScript. Every group counts the same whatever its size, and irrelevance detection — which we also measure, another 240 problems — is not part of it. The margin follows that same weighting.
~159 tok/s on R9700 32GB (Vulkan), measured by us. Q4_K_M GGUF, one request at a time, 16384 context, KV as in the model card. Re-measured 2026-08-29 with our speed kit; the run files (flags, GPU layers, device, llama.cpp build) are in results/speed/kit/r9700/.
Real GGUF file sizes = weight VRAM. Add the KV cache for your context (≈3 GB at 32K, fp16). Read from the GGUF header and checked against what llama.cpp actually allocates.
| Quantization | Size |
|---|---|
| IQ1_S | 8.9 GB |
| IQ1_M | 9.6 GB |
| IQ2_XXS | 10.3 GB |
| IQ2_M | 10.8 GB |
| Q2_K | 11.3 GB |
| Q2_K_L | 11.3 GB |
| Q2_K_XL | 11.8 GB |
| IQ3_XXS | 12.8 GB |
| Q3_K_S | 13.3 GB |
| Q3_K_XL | 13.8 GB |
| Q3_K_M | 14.7 GB |
| IQ4_XS | 16.4 GB |
| IQ4_NL | 17.3 GB |
| Q4_0 | 17.4 GB |
| Q4_K_S | 17.5 GB |
| Q4_K_XL | 17.7 GB |
| Q4_K_M | 18.6 GB |
| Q4_1 | 19.2 GB |
| Q5_K_S | 21.1 GB |
| Q5_K_M | 21.7 GB |
| Q5_K_XL | 21.7 GB |
| Q6_K | 25.1 GB |
| Q6_K_XL | 26.3 GB |
| Q8_0 | 32.5 GB |
| Q8_K_XL | 36.0 GB |
| BF16 | 61.1 GB |
All 30B of experts load into VRAM (~19 GB at Q4) even though only 3B compute — budget for the full size, not the active params. Use YaRN for context beyond its native window. Pairs well with an agent harness (opencode, Cline).