Runs on ≤96 GB · Hugging Face ↗
AA coding index 17.4 (Artificial Analysis, frozen since 2026-09-11)
An 80B-total MoE (~3B active) with a hybrid-attention backbone built for very long context at low active-compute cost.
Real GGUF file sizes = weight VRAM. Add the KV cache for your context (≈3 GB at 32K, fp16). Read from the GGUF header, but this architecture has never been checked against llama.cpp — treat it as an estimate.
| Quantization | Size |
|---|---|
| IQ1_S | 22.9 GB |
| IQ1_M | 24.4 GB |
| IQ2_XXS | 26.2 GB |
| Q2_K | 29.2 GB |
| Q2_K_L | 29.2 GB |
| Q2_K_XL | 30.1 GB |
| IQ3_XXS | 33.1 GB |
| Q3_K_S | 34.6 GB |
| Q3_K_XL | 35.6 GB |
| Q3_K_M | 38.3 GB |
| IQ4_XS | 42.6 GB |
| IQ4_NL | 45.1 GB |
| Q4_0 | 45.3 GB |
| Q4_K_S | 45.5 GB |
| Q4_K_XL | 46.1 GB |
| Q4_K_M | 48.5 GB |
| Q4_1 | 50.1 GB |
| Q5_K_S | 55.0 GB |
| Q5_K_XL | 56.7 GB |
| Q5_K_M | 56.8 GB |
| Q6_K | 65.5 GB |
| Q6_K_XL | 68.5 GB |
| Q8_0 | 84.8 GB |
| Q8_K_XL | 93.1 GB |
| BF16 | 159.5 GB |
~50 GB at Q4 — fits an 80 GB card with lots of room for its long context. The hybrid attention keeps the KV cache smaller than a dense 80B would.