NVIDIA's 2-bit Nemotron 3.5 Bolt 74B fits in 24GB and punches above its weight
Sam Witteveen · youtube · 2026-11-09
Sam Witteveen breaks down NVIDIA's Nemotron-3.5-Bolt-74B-A9B-24GB, a 74B MoE model aggressively quantized to 2 bits to fit in 24GB, performing far better than expected.
The video walks through the full recipe: pruning, post-training, quantization-aware training, and NVIDIA's twist — quantization-aware distillation (QAD) — grounded in the ParetoQ research, explaining why quantization below 4-bit normally breaks and why this pipeline works.
He demos it on a ZGX Fury with SVG generation, agentic tool-calling and concurrency tests, and discusses which hardware Bolt targets. The research report and Hugging Face weights are public.
More from Models
- Open-weights Inkling Small 2 impresses: researcher says he prefers it over DeepSeek V4.1 Flash — gnukeith · 2026-11-13
- Manual Review Exposes How LLMs Cheat Benchmarks: Sandbox Escapes and Copying Peers — teortaxesTex · 2026-11-13
- Models invent their own filler tokens during RL training — code_star · 2026-11-13
- Google Search reportedly gets massive Gemini 4 Hydrogen upgrade with in-search simulations — AI_Andrew · 2026-11-13
- Chinese medical AI beats 250 top doctors 88-12 on complex lung cancer cases in new study — 量子位 · 2026-11-13
- Gemini 4 Carbon ties Argon at 93/98 but burns 2.67× output tokens and 2.22× Python runs — Correct_Tomato1871 · 2026-11-13