Unsloth ships GGUF quants for Nemotron 3.5: Bolt-74B runs in 24GB RAM
danielhanchen · x · 2026-10-15
Unsloth released UD-3 GGUF quants for NVIDIA's newly launched Nemotron 3.5 models, with KLD benchmark data.
Key points:
- Nemotron-3.5-Bolt-74B-A9B: runs on 24GB RAM, top-tier performance for its size, and runs great on RTX Spark
- Nemotron-3.5-Super-VL-120B-A12B: hybrid-reasoning MoE with up to 1M context, accepting text/images/video, supporting JSON output and tool calling; GGUFs span 46.1GB (1-bit, 64GB RAM) to 132GB (8-bit), with 4-bit at 86.9GB (128GB recommended)
- NVFP4 quants (79GB, lossless direct conversion) plus KLD benchmark curves showing divergence across quant levels
More from Infra
- NVIDIA and Sweden's Wallenberg launch Sferical One, a shared AI supercomputer — nvidia · 2026-10-16
- Next infra frontier: self-evolving systems that adapt and self-verify — _shreya_s · 2026-10-16
- Xiaomi's MiMo RL dashboard reveals $3.47M spent on just 30 training steps — bycloud · 2026-10-16
- Cerebras pushes disaggregated inference: heterogeneous prefill pools feeding wafer-scale decode — ai · 2026-10-16
- d-Matrix GA's Corsair rackscale, demos first public Attention-FFN disaggregation at 845 tok/s — ai · 2026-10-16
- Actual's inference engine hits 10k tok/s on NVIDIA DGX Station, GB300 case study — natesiggard · 2026-10-16