NVIDIA Updates 30B Hybrid Model: Distillation Delivers 70% Intelligence Boost
PavloMolchanov · x · 2026-08-11
NVIDIA released a weight update for its smallest 30B-A3B model (MoE+Mamba architecture) from the NVIDIA-Nemotron family. By distilling knowledge from its larger models (Super and Ultra) during continuous pre-training and post-training, the model's capability has significantly improved.
The official AA index jumped from 14 to 24, achieving a 70% performance gain without any architectural changes. This means users can get substantially more intelligence simply by updating the weight file. The new score closely approaches their previous Super model, which is 4x larger. The architecture combines Mamba2 for efficient long-context handling and MoE to save forward compute while maintaining intelligence.
More from Models
- Frontier Models' Biggest Bottleneck is 'Lack of Self-Esteem' in Math Research — coherence · 2026-08-12
- OpenHands Integrates NVIDIA Nemotron to Advance Open, Composable Coding Agents — rajistics · 2026-08-12
- Microsoft's MAI-Image-2.6 Claims No. 2 Spot on Arena, Surpassing Google and Meta — luisdans · 2026-08-12
- Pokee-Isaac 28B Beats Meta and Qwen Rivals in Sub-30B Agent Benchmarks — Kyrannio · 2026-08-12
- Post-Trained NVIDIA Nemotron Beats Claude Opus in Legal Agent Tasks — ctnzr · 2026-08-12
- New LLM Jailbreak Trick: Just Tell It to "Be Smarter Than Grok" — npinto · 2026-08-12