NVIDIA paper: Nemotron-H, a family of hybrid Mamba-Transformer models
cwolferesearch · x · 2026-08-19
NVIDIA released the Nemotron-H paper, detailing a hybrid Mamba-Transformer architecture for accuracy and efficiency. The accompanying discussion highlights midtraining/CPT techniques: emphasizing high-quality domain-specific data (math, code, science) while retaining general data to avoid catastrophic forgetting.
More from Models
- Grok Bot Ran a 12-Hour Data Analysis Nonstop, Handling API Rate Limits Itself — Daniel_Farinax · 2026-10-16
- One year of Recursive Language Models: GPT-5-mini beats GPT-5 by 110% on 132k-token tasks, cheaper too — a1zhang · 2026-10-16
- Users Report Claude Models Suddenly More Compliant After Forced Compaction — RealSharpNinja · 2026-10-16
- Prediction: Opus 5.5-level models on laptops by 2027, backed by Qwen 27B data — cephaloform · 2026-10-16
- Users Say Krea 2 Output Degraded Sharply Since Krea 3 Beta Launched — szywis · 2026-10-16
- LLM-as-a-Verifier lifts Nemotron 3.5 Super by 9-10 points on agentic benchmarks — simonguozirui · 2026-10-16