Runs on ≤8 GB · Hugging Face ↗
Not independently scored by Artificial Analysis — its base model prism-ml/Bonsai-8B-unpacked is the closest proxy
PrismML's 1-bit 8B — ~1.15 GB of weights, small enough to run on a phone. With the patched runtime it is genuinely coherent (not the garbage stock Ollama produces) and surprisingly capable on easy problems for its size.
HumanEval pass@1 63% (19/30), via local GPU (patched llama.cpp, Q1_0). Measured by us on this hardware, as a rough sanity check — HumanEval is a different, easier, partly-contaminated benchmark than Artificial Analysis’ composite, so it is not comparable to the coding-index column.
BigCodeBench-Hard pass@1 6% (7/121), via local GPU (patched llama.cpp, Q1_0 1-bit), comparable full-148. The brutal counterpart to HumanEval — where HumanEval saturates near the top, BCB-Hard spreads the field, so this is the number that actually separates coding ability. The 1-bit 8B holds up on easy HumanEval (63%) but collapses on hard BCB-Hard (7%) — coherent, but the extreme quant shows on hard problems. 160 tok/s, 1.15 GB.
~156 tok/s on RTX 4060 Ti, measured by us.
No GGUF build yet — ~1 GB estimated at 4-bit from the parameter count. Run from safetensors (transformers / vLLM / SGLang).
Needs a 1-bit-capable (PrismML patched) llama.cpp build — not stock Ollama, where it loads but the output is incoherent garbage.
1. Build PrismML's fork (has q1_0 kernels): `cmake -B build -DGGML_CUDA=ON && cmake --build build -j`
2. `build/bin/llama-server -ngl 99 -m Bonsai-8B-Q1_0.gguf -c 8192`
With the right runtime it IS coherent — we measured HumanEval 63% and ~160 tok/s in 1.15 GB. The 1-bit quant only really shows on hard problems (BCB-Hard 7%). A superb size/speed trade for drafts and edge devices.
Tiny footprint and blazing fast (~160 tok/s). Holds up on easy coding (HumanEval 63%) but the 1-bit quant shows on hard problems (BCB-Hard 7%) — a great draft/edge model, not a heavy agentic coder. Needs the 1-bit-capable PrismML build.