Runs on ≤24 GB · Hugging Face ↗
Not independently scored by Artificial Analysis
Released by OpenBMB, Apache-2.0. A vision-language model on the Qwen3.5/3.6 MoE text backbone: 40 layers, three Gated DeltaNet layers for every full-attention one, 256 experts with 8 active plus one shared, about 3B active. SigLIP2 vision encoder. The card claims it beats Qwen3.6-35B-A3B in instruct mode, in less inference time.
Real GGUF file sizes = weight VRAM. Add the KV cache for your context (≈1 GB at 32K, fp16). Read from the GGUF header and checked against what llama.cpp actually allocates.
| Quantization | Size |
|---|---|
| IQ3_XS | 14.5 GB |
| IQ3_M | 15.4 GB |
| IQ4_XS | 18.7 GB |
| Q4_0 | 19.7 GB |
| IQ4_NL | 19.8 GB |
| Q4_K_S | 19.9 GB |
| Q4_K_M | 21.2 GB |
| Q4_1 | 21.8 GB |
| Q5_0 | 24.0 GB |
| Q5_K_S | 24.0 GB |
| Q5_K_M | 24.7 GB |
| Q5_1 | 26.1 GB |
| Q6_K | 28.5 GB |
| Q8_0 | 36.9 GB |
| F16 | 69.4 GB |
The GGUF declares qwen35moe, the architecture of Qwen3.6-35B-A3B, so stock llama.cpp needs no fork for text. OpenBMB's Lookahead Expert Scheduling lives in its own fork (OpenBMB/llama.cpp-omni) and only changes how experts are loaded, not the output. The Q4_K_M is 21.2 GB, so a 24 GB card; the IQ3_M is 15.4 GB. The chat template has an enable_thinking switch. Read it next to the Qwen3.6-35B-A3B card.