NVIDIA's 75B hybrid MoE Nemotron-3-Puzzle is now runnable locally in llama.cpp
jacek2023 · reddit · 2026-09-03
- Model: NVIDIA Nemotron-3-Puzzle-75B-A9B can now run locally via llama.cpp.
- Architecture: Hybrid MoE with interleaved Mamba, MoE, and attention layers; supports Multi-Token Prediction (MTP) for faster generation.
- Specs: Compressed from parent Nemotron-3-Super-120B-A12B (120.7B total / 12.8B active) down to 75.3B total / 9.3B active parameters.
- Why it matters: An interesting size for local deployment — a drop-in option for those who couldn't run Nemotron-3-Super, and worth comparing against the newer Nemotron-3.5-Lightning-30B-A3B.
More from Infra
- Developer Moves AI Agents to a Dedicated Linux Workstation Using Incus System Containers — kevinkern · 2026-10-21
- Nvidia guides 70% growth citing unmet AI demand, while media panics over a $500M Korean neocloud IPO — firstadopter · 2026-10-21
- Pichai: everyone is massively short on AI compute — 'everyone wishes you had two times what you had planned' — firstadopter · 2026-10-21
- Stanford CS336 Lecture 6: fused Triton kernels cut softmax memory traffic ~4x — stanfordnlp · 2026-10-21
- General Compute partners with Positron AI: Atlas memory-centric hardware ships Q1 2027 — aftahi_ai · 2026-10-21
- AI bond issuance hits $419B in 2026, up 90% from all of 2025 — SumitGup · 2026-10-21