Runs on — · Hugging Face ↗
AA coding index 25.3 (Artificial Analysis, frozen since 2026-09-11)
A newer NVIDIA 30B-total MoE (~3B active) using cascade routing — fast decoding for its capability, tuned for the NVIDIA stack.
Real GGUF file sizes = weight VRAM.
| Quantization | Size |
|---|---|
| IQ2_XXS | 18.1 GB |
| IQ2_XS | 18.1 GB |
| IQ2_S | 18.1 GB |
| IQ2_M | 18.1 GB |
| Q2_K | 18.2 GB |
| IQ4_XS | 18.2 GB |
| IQ4_NL | 18.2 GB |
| Q3_K_S | 18.2 GB |
| Q4_0 | 18.3 GB |
| Q2_K_L | 18.3 GB |
| IQ3_XXS | 19.0 GB |
| IQ3_XS | 19.1 GB |
| Q3_K_M | 19.1 GB |
| Q3_K_L | 19.5 GB |
| IQ3_M | 19.5 GB |
| Q3_K_XL | 19.7 GB |
| Q4_1 | 20.1 GB |
| Q4_K_S | 22.4 GB |
| Q5_K_S | 24.0 GB |
| Q4_K_M | 24.7 GB |
| Q4_K_L | 24.9 GB |
| Q5_K_M | 26.2 GB |
| Q5_K_L | 26.3 GB |
| Q6_K | 33.6 GB |
| Q6_K_L | 33.6 GB |
| Q8_0 | 33.6 GB |
| BF16 | 63.2 GB |
~25 GB at Q4 — a 32 GB card, or a 24 GB one with CPU offload of idle experts. vLLM / TensorRT-LLM give the best throughput on NVIDIA hardware.