Thinking Machines Details End-to-End 4-Bit Training: NVFP4 Runs Up to 6x Faster Than BF16 on GB300
clarejtbirch · x · 2026-11-14
Thinking Machines published a deep dive on Connectionism explaining how they trained Inkling 2 Small with a native 4-bit forward pass from the first step of pretraining through the final RL rollout.
Key points:
- The tradeoff: fewer bits mean faster GPU arithmetic and less memory, but fewer representable numbers — rounding-induced quantization errors accumulate across layers and optimization steps and can impair learning.
- Three numerical formats: training mixes BF16, MXFP8, and NVFP4 on NVIDIA's Grace Blackwell Ultra (GB300). NVFP4 stores 3.6x fewer bits than BF16 and runs matrix multiplies up to 6x faster (15 vs 2.5 PFLOPS on GB300).
- NVFP4 acceleration clearly pays off in high-volume inference, but its accuracy cost is much harder to manage in training — the post details how they handled it.
Illustrated with a Monet painting quantized to 16 colors, this is a first-hand training-engineering writeup with real value for low-precision training practice.
Related event: Thinking Machines Details End-to-End 4-Bit Training of Inkling 2 Small(10 posts)→
More from Infra
- SemiAnalysis: Google TPUs match Nvidia in nightly benchmarks; the moat is Jensen's supply chain — elonmusk · 2026-11-14
- Musk warns Nvidia's handshake supply deals may unravel without Jensen — elonmusk · 2026-11-14
- Report: Nvidia veterans may bolt post-Jensen, most already financially free — basedjensen · 2026-11-14
- Jensen Huang calls TSMC chief directly over GPU batch delays, sparking org debate — IgorCarron · 2026-11-14
- Cisco logs strongest revenue growth in 15 years, credits agentic AI demand — Justgototheeffinmoon · 2026-11-14
- NVIDIA and TSMC reportedly never signed a formal contract in nearly 30 years of partnership — IgorCarron · 2026-11-14