Ternary Bonsai 2 27B

Not independently scored by Artificial Analysis — its base model Qwen/Qwen3.8-27B is the closest proxy

Released 2026-09-16 by PrismML, Apache-2.0: the second version of Ternary Bonsai 27B, again built from Qwen3.8-27B with ternary weights. Three builds: PTQ1_0 (6.0 GB), PQ2_0 (7.2 GB) and F16 (53.8 GB).

BigCodeBench-Hard pass@1 27% (40/148), via official protocol · llama.cpp PQ2_0 · one request at a time · a shared cluster GPU · mean of 3 runs, range 0.0 pp. The brutal counterpart to HumanEval — where HumanEval saturates near the top, BCB-Hard spreads the field, so this is the number that actually separates coding ability. repeats itself until it runs out; 2 of 148 answers hit the token limit (effective ceiling 99%)

Two different questions. code is BigCodeBench-Hard: write a working program. tools is BFCL v4 non-live: call the right function with the right arguments. A model can be good at one and ordinary at the other, and quantisation does not hit them the same way — read each column on its own. We have not been able to tie these files back to a published repo, so take the quant names as the labels we ran them under. Every point measured on the same class of card, one pass, one request at a time. One problem is 0.68 points, so anything under ~3.4 points apart is a tie — that is the spread we measure between GPU classes, and our rows are not all on the same card. Each card column is the context left for the KV cache after the weights, KV-Q4 / KV-Q8. A dash means the file does not fit that card, or fits with no room left to work. Context is usually the real constraint, not quality. Computed for one slot (--parallel 1), which is what you get serving yourself. llama-server opens four by default, and on a hybrid that costs real context — the table below each curve shows how much. The card figures assume a dedicated card. Measured on an idle one: the driver keeps 623 MiB of 143,771, so 99.6% is usable. Compute buffers are measured too — a fixed ~130 MiB plus 1 MiB per 1K of context, the same slope on every architecture we checked. If your card also runs your desktop you get noticeably less, and we have not measured that case yet.

One request at a time, same server build and same KV cache throughout. Every point on the same class of card. Points measured once carry the sampling margin only; where the curve jumps we measure again and give the spread. A row is only here if all seven categories completed.

All 13 RULER tasks, one request at a time, KV cache f16, bench commit c3f5e3b4, seed 42. The score is the mean of the 13; the margin is two standard deviations of the sampling, so it grows as the sample shrinks and as the score drops.

This is not a coding score and must not be read as one. RULER asks whether the model finds and follows a fact buried in a long text. On this model PTQ1_0 scores 72.3 at 32K against 71.8 for PQ2_0, a difference of 0.5 points — while on BigCodeBench-Hard the same two files score 25.7% and 27.0%. A page that read the long-context number as a general score would be telling the truth and misleading you.

Stock llama.cpp will not run it: it rejects PQ2_0 and PTQ1_0 as unknown types. The weights are stored in a rotated basis, so the runtime has to apply a Hadamard rotation to the activations, and only PrismML's own llama.cpp fork does that, on CUDA, Metal or CPU. It does not run on AMD cards through Vulkan, and vLLM does not apply the rotation either. We measured it with that fork.