Replicating 4-bit forward-pass training on one B200: 38% faster at 3.3B with identical loss
OtherRaisin3426 · reddit · 2026-11-17
After Thinking Machines said Inkling 2 Small was trained with a native 4-bit forward pass, a Redditor replicated it end to end on a single B200 (Modal, PyTorch 2.8, cuBLASLt NVFP4/FP8 via ctypes): forward in NVFP4, backward in FP8 with stochastic rounding, master weights in high precision. Results: 4-bit training ran faster (840M: 5.5 min → 4.3 min for 2,000 steps), with bigger models gaining more (18%/28%/38% for 219M/840M/3.3B), and final validation loss was essentially identical (1.69 vs 1.691). Gotchas: Blackwell's convert instruction rounds ties toward zero, breaking bit-for-bit checks, and PyTorch's attention backward isn't bit-repeatable. Full write-up, code, and a 3-hour video are linked.
More from Infra
- India's sovereign AI lab Sarvam hosts mostly Chinese models on subsidized H100s — ayushthakur0 · 2026-11-17
- DeepSeek limits usage over compute shortages, raising doubts about its free service sustainability — Brave-Experience3228 · 2026-11-17
- Nvidia put Megaspeed on its 'no sell' list in June 2025 after US investigators requested sales records — Polymarket · 2026-11-17
- NVIDIA open-sources cuPotential, over 3x faster MD nonbonded interactions vs OpenMM — AllThingsApx · 2026-11-17
- Syzygy says compute shortage is a software problem, claiming 6% to 90% GPU utilization gains — teortaxesTex · 2026-11-17
- Commerce and DOJ issue joint statement on Megaspeed chip smuggling probe; Nvidia added firm to no-sell list — kevinsxu · 2026-11-17