Runs on ≤8 GB · Hugging Face ↗
Coding index 28.7 (Artificial Analysis)
A dense 9B with a thinking mode — punches above its size on reasoning for something that fits an 8 GB card comfortably.
HumanEval pass@1 40% (12/30), via local GPU (Ollama, Q4_K_M). Measured by us on this hardware, as a rough sanity check — HumanEval is a different, easier, partly-contaminated benchmark than Artificial Analysis’ composite, so it is not comparable to the coding-index column.
BigCodeBench-Hard pass@1 12% (14/121), via local GPU (Ollama, num_ctx 16384, budget 12K). The brutal counterpart to HumanEval — where HumanEval saturates near the top, BCB-Hard spreads the field, so this is the number that actually separates coding ability. Comparable full-148 at 16K context, but a very verbose reasoner — 56/148 still hit the 12K budget, so a near-floor.
~44 tok/s on RTX 4060 Ti, measured by us.
Real GGUF file sizes = weight VRAM.
| Quantization | Size |
|---|---|
| Q2_K | 4.1 GB |
| Q3_K_M | 4.7 GB |
| IQ4_XS | 5.2 GB |
| Q4_0 | 5.4 GB |
| Q4_K_S | 5.4 GB |
| Q4_K_M | 5.7 GB |
| Q4_1 | 5.8 GB |
| Q4 | 6.0 GB |
| Q5_K_M | 6.6 GB |
| Q8_0 | 9.5 GB |
| Q8 | 13.0 GB |
| Q6_K | 16.2 GB |
| BF16 | 17.9 GB |
Fits well under 8 GB at Q4. Toggle reasoning with the /think and /no_think controls; disable it for fast, cheap autocomplete-style use.