Runs on ≤8 GB · Hugging Face ↗
Not independently scored by Artificial Analysis — its base model Qwen/Qwen3.6-27B is the closest proxy
A ternary (1.71 bits-per-weight) rebuild of Qwen3.6-27B — a 27B-class model whose weights shrink from ~54 GB to ~7 GB. PrismML reports it keeps ~94.6% of the full-precision quality (their benchmark; unverified by an independent lab).
HumanEval pass@1 90% (18/20), via local GPU (patched llama.cpp, Q2_0), 20K budget. Measured by us on this hardware, as a rough sanity check — HumanEval is a different, easier, partly-contaminated benchmark than Artificial Analysis’ composite, so it is not comparable to the coding-index column.
BigCodeBench-Hard pass@1 36% (43/121), via local GPU (patched llama.cpp, Q2_0 ternary), comparable full-148, 24K ctx / 20K budget. The brutal counterpart to HumanEval — where HumanEval saturates near the top, BCB-Hard spreads the field, so this is the number that actually separates coding ability. Comparable full-148 (only 2/148 truncated). A ternary 27B in 7 GB that lands with the strongest local coders — verbose but capable.
~30 tok/s on RTX 4060 Ti, measured by us.
Real GGUF file sizes = weight VRAM.
| Quantization | Size |
|---|---|
| Q4_1 | 1.9 GB |
| Q2_0 | 7.2 GB |
| BF16 | 7.3 GB |
| F16 | 53.8 GB |
Stock llama.cpp / Ollama / LM Studio will load the file but emit garbage — the ternary (Q2_0, ~1.71 bpw) weights need custom kernels that only exist in PrismML's patched fork.
1. `git clone https://github.com/PrismML-Eng/llama.cpp && cd llama.cpp`
2. `cmake -B build -DGGML_CUDA=ON && cmake --build build -j`
3. `build/bin/llama-server -ngl 99 -m Ternary-Bonsai-27B-Q2_0.gguf -c 24576`
With the patched runtime it is fully coherent and generates correct code at ~31 tok/s on an RTX 4060 Ti — a 27B in ~7 GB. It's a reasoning model, so give it a large context and a high token budget: it emits ~2,000 thinking tokens even for a trivial prompt, and overran a 6K budget on the harder BigCodeBench problems.
Use the ternary GGUF (~7 GB), not the tiny "dspark" Q4_1 which trades away more quality. Ternary kernels need a recent llama.cpp build. Weights fit an 8 GB card, but the 262K context is the real cost — a big KV cache is what fills VRAM, so cap context and consider a quantized KV cache.