Runs on ≤16 GB · Hugging Face ↗
Not independently scored by Artificial Analysis
AMD's fully-open MoE reasoner — 16B total, ~3B active across 64 experts on a DeepSeek-V3-style architecture, with a visible think phase. A rare frontier-style open release straight from AMD; trending on HF but unrated by Artificial Analysis.
Real GGUF file sizes = weight VRAM. Add the KV cache for your context (≈7 GB at 32K, fp16).
| Quantization | Size |
|---|---|
| Q2_K | 6.5 GB |
| Q3_K_M | 8.2 GB |
| Q4_K_M | 10.5 GB |
| Q5_K_M | 12.0 GB |
| Q6_K | 14.2 GB |
| Q8_0 | 16.9 GB |
No GGUF converter yet for the `InstellaMoE` architecture, so llama.cpp / Ollama / LM Studio won't load it, and vLLM 0.26 also fails on it (custom arch not recognised — confirmed 2026-07-31). Serve with transformers (bf16) — confirmed loading on 2026-08-06 (37 GB resident on RTX PRO 6000 96 GB, ~17 tok/s), but a full BCB-148 run would take ~10–13 h (~$12–15), so we didn't complete it. Needs a faster card or a vLLM build that recognises the arch to be worth measuring.
~10 GB at Q4 in principle, but no GGUF exists yet — the custom `InstellaMoE` arch is transformers-only for now (like Nanbeige). The small active set keeps it fast once loaded; a floor until we run it first-party.