Runs on ≤96 GB · Hugging Face ↗
AA coding index 45.7 (Artificial Analysis, frozen since 2026-09-11)
A 122B-total MoE (~10B active) — the strongest model that fits a single 80 GB card, and far above the other 80 GB-class open models on coding.
Real GGUF file sizes = weight VRAM. Add the KV cache for your context (≈1 GB at 32K, fp16). Read from the GGUF header and checked against what llama.cpp actually allocates.
| Quantization | Size |
|---|---|
| IQ1_M | 38.7 GB |
| IQ2_XXS | 40.3 GB |
| IQ2_M | 40.4 GB |
| Q2_K_XL | 42.8 GB |
| IQ3_XXS | 47.8 GB |
| IQ3_S | 52.0 GB |
| Q3_K_M | 58.2 GB |
| Q3_K_XL | 58.4 GB |
| IQ4_XS | 61.9 GB |
| IQ4_NL | 63.2 GB |
| Q4_K_S | 73.4 GB |
| MXFP4 | 76.2 GB |
| Q4_K_M | 78.3 GB |
| Q4_K_XL | 78.6 GB |
| Q5_K_S | 88.3 GB |
| Q5_K_M | 93.6 GB |
| Q5_K_XL | 93.9 GB |
| Q6_K | 104.1 GB |
| Q6_K_XL | 113.9 GB |
| Q8_0 | 132.6 GB |
| Q8_K_XL | 134.9 GB |
| BF16 | 249.4 GB |
~78 GB at Q4 on one 80 GB card (H100 / A100), or a 2-GPU split with headroom. The ~10B active params keep decoding brisk despite the size.