Ornith-1.0 35B

Not independently scored by Artificial Analysis

The 35B-total MoE middle sibling of DeepReinforce's self-scaffolding agentic coder (MIT), a Qwen3.5-MoE. A genuine step up from the 9B on our first-party BCB-Hard, and among the stronger open coders the AA index doesn't rank. Agentic by design, so a single-shot benchmark is a floor for its agent-loop behaviour.

BigCodeBench-Hard pass@1 26% (39/148), via official protocol · llama.cpp Q4_K_M · one request at a time · a shared cluster GPU · mean of 3 runs, range 0.0 pp. The brutal counterpart to HumanEval — where HumanEval saturates near the top, BCB-Hard spreads the field, so this is the number that actually separates coding ability.

86.1% ± 2.2 over 1150 problems

Q4_K_M · a shared cluster GPU · KV f16 · 1 run · spread between runs assumed at 0.25 points, not measured. This is the non-agentic half of BFCL: does the model call the right function with the right arguments. It says nothing about how it behaves over a long agentic conversation, which is a separate measurement. The score is BFCL’s own: the unweighted mean of four groups (simple, multiple, parallel, parallel multiple), where simple is itself the mean of Python, Java and JavaScript. Every group counts the same whatever its size, and irrelevance detection — which we also measure, another 240 problems — is not part of it. The margin follows that same weighting.

All 13 RULER tasks, one request at a time, KV cache f16, bench commit c3f5e3b4, seed 42. The score is the mean of the 13; the margin is two standard deviations of the sampling, so it grows as the sample shrinks and as the score drops.

This is not a coding score and must not be read as one. RULER asks whether the model finds and follows a fact buried in a long text. Quantising hits the two differently, so a model that still retrieves at length may already have lost the code.

~107 tok/s on R9700 32GB (Vulkan), measured by us. Q4_K_M GGUF, one request at a time, 16384 context, KV as in the model card. Re-measured 2026-08-29 with our speed kit; the run files (flags, GPU layers, device, llama.cpp build) are in results/speed/kit/r9700/.

~20 GB at Q4 — needs a 24 GB+ card or the 32 GB tier; the MoE keeps decode fast. We benchmarked it on a rented RTX PRO 6000 Blackwell 96 GB (vLLM, BF16) since it doesn't fit a 16 GB card.