Runs on Data centre · Hugging Face ↗
Coding index 22.1 (Artificial Analysis)
A 235B-total MoE (~22B active) — a widely-served open flagship, Apache-2.0.
Real GGUF file sizes = weight VRAM. Add the KV cache for your context (≈6 GB at 32K, fp16).
| Quantization | Size |
|---|---|
| Q3_K_M | 112.4 GB |
| IQ4_XS | 125.5 GB |
| Q4_0 | 133.1 GB |
| Q4_K_S | 133.7 GB |
| Q4 | 134.3 GB |
| Q4_K_M | 142.2 GB |
| Q4_1 | 147.2 GB |
| Q5_K_M | 166.8 GB |
| Q8_0 | 249.9 GB |
| Q2_K | 260.3 GB |
| Q8 | 274.3 GB |
| Q6_K | 395.0 GB |
| BF16 | 470.3 GB |
~142 GB at Q4 — multi-GPU tensor-parallel. All experts must be resident even though only ~22B compute. A common self-host target for those with two 80 GB cards.