Runs on Data centre · Hugging Face ↗
Coding index 68.8 (Artificial Analysis)
Z.ai's flagship — a very large Mixture-of-Experts (hundreds of billions of total params). One of the strongest open models by coding index, but firmly data-centre scale to run locally, even at Q4 (hundreds of GB).
Real GGUF file sizes = weight VRAM. Add the KV cache for your context (≈126 GB at 32K, fp16).
| Quantization | Size |
|---|---|
| Q2_K | 253.9 GB |
| Q3_K_M | 342.7 GB |
| IQ4_XS | 365.3 GB |
| Q4_K_S | 436.4 GB |
| Q4_K_M | 465.8 GB |
| Q4 | 467.3 GB |
| Q5_K_M | 560.8 GB |
| Q8_0 | 801.4 GB |
| Q8 | 819.7 GB |
| Q6_K | 1310.2 GB |
| BF16 | 1508.0 GB |
Multi-GPU or a serious server only — a Q4 build is still hundreds of GB of weights plus a large KV cache. Serve with vLLM or SGLang tensor-parallel across cards. For desktops it is far cheaper to rent it per token (see the leaderboard).