Thinking Machines details end-to-end native 4-bit training of Inkling 2 Small
thinkymachines · x · 2026-11-14
Thinking Machines Lab published a detailed writeup on training Inkling 2 Small with a native 4-bit forward pass from the first pretraining step through the final RL rollout.
Core tension: fewer bits means faster, cheaper arithmetic, but quantization rounding errors accumulate across layers and optimization steps and can impair learning.
Approach: training runs on NVIDIA's Grace Blackwell Ultra (GB300), mixing three numerical formats:
- NVFP4: 4.5 bits per value, stores 3.6x fewer bits than BF16 and runs up to 6x faster (15 vs 2.5 PFLOPS on GB300)
- MXFP8: 8.25 bits, 8.5 PFLOPS
- BF16: 16 bits, 2.5 PFLOPS
The post uses a Monet painting quantized to 16 colors as an analogy for quantization error, and discusses how the speed/accuracy tradeoff differs between high-volume inference and training.
More from Infra
- Quantized all-to-all: NVFP4 dispatch plus RHT+S2F6 8-bit combine to fix MoE comms bottlenecks — snowclipsed · 2026-11-14
- Jensen Huang's exit could trigger a credit crunch across Wall Street's $500B AI financing bet — minhnguyen · 2026-11-14
- Quantizing training to MXFP8 with stochastic rounding: the accuracy-speed sweet spot — snowclipsed · 2026-11-14
- NVIDIA lands nvfp4 in large-scale pretraining under codename "yolo bundle" — SonglinYang4 · 2026-11-14
- Drogue raises $20M to power floating data centers with the ocean's heat — arian_ghashghai · 2026-11-14
- Megaspeed was NVIDIA's largest chip buyer in Southeast Asia by 2025, now official cloud partner — RileyRalmuto · 2026-11-14