Runs on ≤32 GB · Hugging Face ↗
Coding index 41.9 (Artificial Analysis)
A 35B-total MoE with ~3B active — fast decoding for its capability, but all 35B of experts load into VRAM.
BigCodeBench-Hard pass@1 33% (40/121), via rented RTX PRO 6000 Blackwell 96 GB (vLLM, BF16) — comparable full-148, conc64. The brutal counterpart to HumanEval — where HumanEval saturates near the top, BCB-Hard spreads the field, so this is the number that actually separates coding ability. A verbose reasoner (~4.5K tokens/problem, finish=stop — not truncated). Punches above its AA index.
Real GGUF file sizes = weight VRAM.
| Quantization | Size |
|---|---|
| Q2_K | 12.3 GB |
| Q3_K_M | 16.6 GB |
| IQ4_XS | 17.7 GB |
| Q4_K_S | 20.9 GB |
| MXFP4 | 21.7 GB |
| Q4_K_M | 22.1 GB |
| Q4 | 22.4 GB |
| Q5_K_M | 26.5 GB |
| Q8_0 | 36.9 GB |
| Q8 | 38.5 GB |
| Q6_K | 61.2 GB |
| BF16 | 69.4 GB |
~21 GB at Q4; plan for the full size, not the 3B active. MoE offload to CPU lets a 24 GB card run it slower if you are short on VRAM.