Thinking Machines details fully native 4-bit training for Inkling 2 Small

Thinking Machines Lab published a technical blog post, "End-to-End 4-Bit Training" (by Jack Cook and team), offering the first systematic look at the low-precision training recipe behind Inkling 2 Small: from the very first pretraining step to the final RL rollout, forward computation runs entirely in native 4-bit. This tackles the core tension in low-bit training—faster and more memory-efficient, but quantization rounding errors hurt accuracy.

Confirmed

The expert layers in the forward pass use NVFP4 precision with dedicated kernels; backpropagation uses MXFP8 precision, combined with requantization and stochastic rounding.
Figures in the blog show that at scale, this recipe delivers roughly an 8% wall-clock speedup in pretraining and a 20% speedup in RL compared with full MXFP8 training.
Quantized checkpoints ship without requiring calibration.
The recipe draws on prior work: the last 16.6% of layers are computed at higher precision (MXFP8 rather than BF16); activation quantization uses the Four Over Six (4/6) technique to choose between two scaling factors.

Unconfirmed

@snowclipsed questioned whether the recipe uses MXFP8 in the expert layers; the blog materials contain no explicit answer.

Why it matters

@snowclipsed noted that the "NVFP4 forward, MXFP8 backward with stochastic rounding" decision is a small surprise compared with other NVFP4 approaches but very sensible, showing that 4-bit training is now engineering-viable end to end.
@rohanpaulai pointed out that low-precision (e.g., 4-bit) compute can be up to 6x faster than BF16; if the recipe generalizes, it could significantly cut compute costs for large-scale pretraining and RL.

2026-11-14 ~ 2026-11-14 · 5 related posts

Primary sources

Thinking Machines details end-to-end native 4-bit training of Inkling 2 Small — thinkymachines · 2026-11-14 05:23 (UTC+08:00)
Thinking Machines details end-to-end 4-bit training: NVFP4 up to 6x faster than BF16 on GB300 — rohanpaul_ai · 2026-11-14 07:32 (UTC+08:00)

[source] Thinking Machines details end-to-end native 4-bit training of Inkling 2 Small — thinkymachines · 2026-11-14
NVFP4 forward, MXFP8 backward with stochastic rounding: how Inkling 2 Small was trained — snowclipsed · 2026-11-14
8% pretraining and 20% RL speedup from native 4-bit NVFP4 training, screenshots show — snowclipsed · 2026-11-14
Thinking Machines details its NVFP4 training recipe: per-expert scales, 4/6 activation quantization — snowclipsed · 2026-11-14
[source] Thinking Machines details end-to-end 4-bit training: NVFP4 up to 6x faster than BF16 on GB300 — rohanpaul_ai · 2026-11-14