Runs on ≤8 GB · Hugging Face ↗
Coding index 9.5 (Artificial Analysis)
IBM's enterprise-focused 8B, Apache-2.0 licensed, tuned for tool use and RAG rather than chat flair.
HumanEval pass@1 67% (20/30), via local GPU (Ollama, Q4_K_M). Measured by us on this hardware, as a rough sanity check — HumanEval is a different, easier, partly-contaminated benchmark than Artificial Analysis’ composite, so it is not comparable to the coding-index column.
BigCodeBench-Hard pass@1 17% (21/121), via local GPU (Ollama, Q4_K_M). The brutal counterpart to HumanEval — where HumanEval saturates near the top, BCB-Hard spreads the field, so this is the number that actually separates coding ability.
~46 tok/s on RTX 4060 Ti, measured by us.
Real GGUF file sizes = weight VRAM. Add the KV cache for your context (≈5 GB at 32K, fp16).
| Quantization | Size |
|---|---|
| Q2_K | 3.4 GB |
| Q3_K_M | 4.3 GB |
| Q4_0 | 5.1 GB |
| Q4_K_S | 5.1 GB |
| Q4_K_M | 5.3 GB |
| Q4_1 | 5.6 GB |
| Q5_K_M | 6.3 GB |
| Q6_K | 7.2 GB |
| Q8_0 | 9.3 GB |
| BF16 | 17.6 GB |
~5-6 GB at Q4 with room for long context. A safe, permissively-licensed default for on-prem / commercial use where model licence matters.