Runs on Multi-GPU · Hugging Face ↗
AA coding index 48.2 (Artificial Analysis, frozen since 2026-09-11)
A 397B-total MoE (~17B active) — strong quality, data-centre footprint.
Real GGUF file sizes = weight VRAM. Add the KV cache for your context (≈2 GB at 32K, fp16). Read from the GGUF header and checked against what llama.cpp actually allocates.
| Quantization | Size |
|---|---|
| IQ1_M | 106.8 GB |
| IQ2_XXS | 114.9 GB |
| IQ2_M | 123.1 GB |
| IQ3_XXS | 140.3 GB |
| IQ3_S | 146.4 GB |
| Q3_K_S | 164.3 GB |
| Q3_K_M | 177.4 GB |
| Q3_K_XL | 178.7 GB |
| IQ4_XS | 189.7 GB |
| IQ4_NL | 193.8 GB |
| Q4_K_S | 228.0 GB |
| MXFP4 | 237.4 GB |
| Q4_K_M | 244.1 GB |
| Q4_K_XL | 245.3 GB |
| Q5_K_S | 276.6 GB |
| Q5_K_M | 293.7 GB |
| Q5_K_XL | 294.9 GB |
| Q6_K | 326.6 GB |
| Q6_K_XL | 362.3 GB |
| Q8_0 | 421.5 GB |
| Q8_K_XL | 427.7 GB |
| BF16 | 793.0 GB |
~260 GB at Q4, multi-GPU tensor-parallel. The ~17B active params make decoding faster than the total size implies, but all experts must be resident.