Runs on — · Hugging Face ↗
Not independently scored by Artificial Analysis — its base model nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16 is the closest proxy
Released by NVIDIA under OpenMDW-1.1, which allows commercial use. A vision-language model compressed from Nemotron 3.5 Super VL 120B-A12B: 74B total, about 9B active. Hybrid Mamba-2 and Transformer, 88 layers: 40 Mamba-2, 40 MoE and 8 attention. Each MoE layer keeps between 240 and 496 routed experts plus one shared, with 9 active per token (22 in the parent). NVIDIA trains it at 2 bits: the experts are IQ2_XXS with quantization-aware distillation, so the 2-bit file is the model and there is no other quant to compare.
IQ2 GGUF, official from NVIDIA and the only quant (22.0 GB, plus a 1.7 GB mmproj for images): 22.0 GB on disk.
The single GGUF is 22.0 GB. NVIDIA's own llama.cpp recipe for a 32 GB card uses 65,536 tokens of context with a q8_0 KV cache; the full 262,144 needs a 48 GB card. Same llama.cpp architecture as Nemotron 3.5 Lightning (nemotron_h_moe), whose KV-cache estimate we measured 33-80% too high, so this site does not say which cards it fits yet. Thinking is switched with enable_thinking in the chat template; NVIDIA recommends temperature 1.0 and top_p 0.95.