Runs on ≤32 GB · Hugging Face ↗
Not independently scored by Artificial Analysis
A Mixture-of-Experts coder — 30B total but only ~3B active per token, so it runs faster than its size suggests while all experts must still sit in VRAM. Purpose-built for agentic / tool-use coding.
HumanEval pass@1 98% (39/40), via OpenRouter. Measured by us on this hardware, as a rough sanity check — HumanEval is a different, easier, partly-contaminated benchmark than Artificial Analysis’ composite, so it is not comparable to the coding-index column.
BigCodeBench-Hard pass@1 42% (14/33), via OpenRouter (also runs locally). The brutal counterpart to HumanEval — where HumanEval saturates near the top, BCB-Hard spreads the field, so this is the number that actually separates coding ability.
~40 tok/s on RTX 4060 Ti, measured by us. Q4_K_M on an RTX 4060 Ti; 18 GB spills past 16 GB so it partly offloads, but the A3B MoE keeps it fast.
Real GGUF file sizes = weight VRAM. Add the KV cache for your context (≈3 GB at 32K, fp16).
| Quantization | Size |
|---|---|
| Q3_K_M | 14.7 GB |
| IQ4_XS | 16.4 GB |
| Q4_0 | 17.4 GB |
| Q4_K_S | 17.5 GB |
| Q4 | 17.7 GB |
| Q4_K_M | 18.6 GB |
| Q4_1 | 19.2 GB |
| Q5_K_M | 21.7 GB |
| Q8_0 | 32.5 GB |
| Q2_K | 34.4 GB |
| Q8 | 36.0 GB |
| Q6_K | 51.4 GB |
| BF16 | 61.1 GB |
All 30B of experts load into VRAM (~19 GB at Q4) even though only 3B compute — budget for the full size, not the active params. Use YaRN for context beyond its native window. Pairs well with an agent harness (opencode, Cline).