Inkling 2 Small's Native 4-Bit Training: Debut and Doubts

Episode 1 · Thinking Machines Details End-to-End 4-Bit Training of Inkling 2 Small (2026-11-14, 10 posts)

Thinking Machines Lab published a technical blog post, "End-to-End 4-Bit Training" (by Jack Cook and team), offering the first systematic look at the low-precision training recipe behind Inkling 2 Small: from the very first pretraining step to the final RL rollout, forward computation runs entirely in native 4-bit. This tackles the core tension in low-bit training—faster and more memory-efficient, but quantization rounding errors hurt accuracy.

Confirmed

• The expert layers in the forward pass use NVFP4 precision with dedicated kernels; backpropagation uses MXFP8 precision, combined with requantization and stochastic rounding.
• Figures in the blog show that at scale, this recipe delivers roughly an 8% wall-clock speedup in pretraining and a 20% speedup in RL compared with full MXFP8 training.
• Quantized checkpoints ship without requiring calibration.
• The recipe draws on prior work: the last 16.6% of layers are computed at higher precision (MXFP8 rather than BF16); activation quantization uses the Four Over Six (4/6) technique to choose between two scaling factors.

Unconfirmed

• @snowclipsed questioned whether the recipe uses MXFP8 in the expert layers; the blog materials contain no explicit answer.

Why it matters

• @snowclipsed noted that the "NVFP4 forward, MXFP8 backward with stochastic rounding" decision is a small surprise compared with other NVFP4 approaches but very sensible, showing that 4-bit training is now engineering-viable end to end.
• @rohanpaulai pointed out that low-precision (e.g., 4-bit) compute can be up to 6x faster than BF16; if the recipe generalizes, it could significantly cut compute costs for large-scale pretraining and RL.

• Thinking Machines details end-to-end native 4-bit training of Inkling 2 Small — thinkymachines · 2026-11-14
• Inkling 2 Small trained partly in NVFP4 with mixed-precision recipe — HanGuo97 · 2026-11-14
• NVFP4 forward, MXFP8 backward with stochastic rounding: how Inkling 2 Small was trained — snowclipsed · 2026-11-14
• 8% pretraining and 20% RL speedup from native 4-bit NVFP4 training, screenshots show — snowclipsed · 2026-11-14
• Thinking Machines details end-to-end native 4-bit training of Inkling 2 Small on GB300 — clarejtbirch · 2026-11-14
• Thinking Machines Details End-to-End 4-Bit Training: NVFP4 Runs Up to 6x Faster Than BF16 on GB300 — clarejtbirch · 2026-11-14
• Thinking Machines details its NVFP4 training recipe: per-expert scales, 4/6 activation quantization — snowclipsed · 2026-11-14
• Thinking Machines details end-to-end 4-bit training: NVFP4 up to 6x faster than BF16 on GB300 — rohanpaul_ai · 2026-11-14
• Thinking Machines trains 594B Inkling 2 Small in 68 hours with native 4-bit — jonlachman · 2026-11-14
• Thinking Machines details training Inkling 2 Small with native 4-bit forward passes — clarejtbirch · 2026-11-14

Episode 2 · Researchers Question Payoff of Inkling 2 Small's Native 4-bit Training (2026-11-14, 2 posts)

Andrew Carr and Nathan Lambert question the value of Inkling 2 Small's native 4-bit end-to-end training, noting it only sped up pretraining by about 8% (roughly five hours), which they argue hardly justifies the engineering effort.

• Native 4-bit end-to-end training yields just 8% pretraining speedup, sparking debate — andrew_n_carr · 2026-11-14
• Nat Lambert questions: was all that work worth just a 5-hour pre-training speedup? — natolambert · 2026-11-14