NVIDIA ships Nemotron 3.5 Bolt: 74B MoE agent model compressed into 24 GB
lmoroney · x · 2026-10-16
NVIDIA released Nemotron 3.5 Bolt on Hugging Face, aiming to run a 74B-parameter vision-language agent model in 24 GB of memory.
Key points:
- MoE with 9B active of 74B total, compressed from the 120B Nemotron 3.5 Super VL
- Experts quantized to 2 bits with quantization-aware distillation; Mamba layers kept in NVFP4, embedding/output layers at 8 bits
- Accepts text, images, and video with up to 256K context; tuned for tool calling, browsing, and terminal work
- NVIDIA reports 51.0 on Terminal-Bench 2.1 vs 44.4 for Qwen3.6-35B-A3B in BF16 (vendor numbers)
- OpenMDW 1.1 license, with support in vLLM, llama.cpp, Ollama, and LM Studio
The author also suggests it as a teaching case study: have students estimate what each compression step from 120B to 24 GB saved.
Related event: NVIDIA Launches Nemotron 3.5 Bolt: 74B Model in 24GB VRAM(2 posts)→
More from Infra
- EpochAI pegs GPT-6 training at up to 1.8e27 FLOP; scaling01 calls param estimates way too high — scaling01 · 2026-10-16
- Meta's Muse Spark Open scores 39 vs 48 for closed Spark 1.3 on AA Intelligence Index — EAccelerate_42 · 2026-10-16
- LLM spots useless 95GB-per-frame memcpy in ffmpeg dblur, cutting runtime from 2452ms to 49ms — IgorCarron · 2026-10-16
- Tree packing lands in TRL's AsyncGRPO, cutting agentic RL training compute by over 26x — joecole · 2026-10-16
- ASM's pizza-themed ALD explainer at SEMICON West is the best yet — BenBajarin · 2026-10-16
- Google's Willow chip cuts quantum errors 40x, raising Q-Day concerns for Bitcoin — jamestagg · 2026-10-16