Runs on Data centre · Hugging Face ↗
Coding index 46.9 (Artificial Analysis)
Mistral's 128B mid-flagship — the best coder in the batch that gets close to fitting a single big card rather than needing a cluster.
Real GGUF file sizes = weight VRAM.
| Quantization | Size |
|---|---|
| Q3_K_M | 60.6 GB |
| IQ4_XS | 67.1 GB |
| Q4_0 | 71.0 GB |
| Q4_K_S | 71.2 GB |
| Q4_K_M | 74.9 GB |
| Q4 | 75.7 GB |
| Q4_1 | 78.5 GB |
| Q5_K_M | 88.3 GB |
| Q8_0 | 132.9 GB |
| Q2_K | 141.7 GB |
| Q8 | 144.7 GB |
| Q6_K | 211.7 GB |
| BF16 | 250.1 GB |
~75 GB at Q4 — a single 80 GB card at a tight quant / short context, or a 2-GPU split. The practical ceiling for one-node self-hosting among these.