Runs on — · Hugging Face ↗
AA coding index 37.7 (Artificial Analysis, frozen since 2026-09-11)
NVIDIA's 120B-total MoE (~12B active) — the best measured model in the 80 GB tier, tuned to run well on NVIDIA's own stack.
Real GGUF file sizes = weight VRAM.
| Quantization | Size |
|---|---|
| IQ1_M | 52.7 GB |
| IQ2_XXS | 52.7 GB |
| IQ2_M | 52.7 GB |
| Q2_K_XL | 54.7 GB |
| IQ3_S | 56.6 GB |
| IQ3_XXS | 56.6 GB |
| Q3_K_M | 61.7 GB |
| Q3_K_S | 61.7 GB |
| Q3_K_XL | 62.6 GB |
| IQ4_NL | 64.5 GB |
| IQ4_XS | 64.5 GB |
| Q4_K_S | 79.0 GB |
| MXFP4 | 82.1 GB |
| Q4_K_M | 82.5 GB |
| Q4_K_XL | 83.8 GB |
| Q5_K_S | 89.7 GB |
| Q5_K_M | 107.3 GB |
| Q5_K_XL | 107.6 GB |
| Q6_K | 114.8 GB |
| Q6_K_XL | 117.9 GB |
| Q8_0 | 128.5 GB |
| Q8_K_XL | 132.5 GB |
| BF16 | 241.5 GB |
~79 GB at Q4 — a single 80 GB card or a 2-GPU split. vLLM / TensorRT-LLM give the best throughput on NVIDIA hardware.