Crowdsourced solvers make NVIDIA's Nemotron 3.5 Lightning 34.6% faster on Mac
gajesh · x · 2026-10-24
NVIDIA's open-weight Nemotron 3.5 Lightning (20GB, designed for local single-machine use) is the target of a community speed-up competition. Because NVIDIA built it as a fast general base model for post-training, any base speedup carries over to every fine-tune on top.
Key points:
- On day one, solvers on Yukon Research got it 31.9% faster on DGX Spark and 13.8% faster on Mac
- Current record: 34.6% faster overall on Mac — decode from 152.3 to 237.3 tok/s, prefill 3220.7 tok/s, via a custom draft head for speculative decoding
- Optimizable surfaces include Mamba-2, mixture-of-experts, attention layers, and a multi-token prediction head
- NVIDIA also released training data and recipes for community post-training
More from Infra
- Top law firm reportedly ditches Harvey and cloud APIs, self-hosts Kimi and DeepSeek — SumitGup · 2026-10-24
- Tesla Energy Q3: deployments up 10% but revenue down 16% as prices collapse — pdamodaran · 2026-10-24
- Mandiant: Some PHP Vulnerability PoCs Achieve RCE on mod_php, Patch Now — cyb3rops · 2026-10-24
- humans& W4A16 kernel hits 98% Tensor Core utilization, 10% faster than FlashInfer NVFP4 — niloofar_mire · 2026-10-24
- Chanos questions AI compute boom as Blackstone shops data centers while neocloud rates jump 50% — GaryMarcus · 2026-10-24
- Cancer center's $300-400K GPU dilemma: RTX Pro 5000 pairs vs one 8-GPU box for AlphaFold — starkruzr · 2026-10-24