Thinking Machines' Inkling 2 Small trained on 12.55T tokens, with RL consuming over a quarter of pretraining compute
YouJiacheng · x · 2026-11-06
YouJiacheng does the math on Thinking Machines' (TML) Inkling 2 Small training: the four pretraining phases total 8.4T+3.8T+0.32T+0.034T ≈ 12.55T tokens, and the main RL run used 3T+ tokens — more than a quarter of the pretraining compute (previous estimates said 1/10).
Training details from the screenshot:
- H1: 8.4T tokens of mixed natural language, code, and image data; 68 hours on NVIDIA GB300 NVL72 racks; natively trained in NVFP4 4-bit format.
- H2: 3.8T tokens of higher-quality unseen data at 512K context.
- H3: 320B tokens of reasoning and agent trajectories at 1M context, mixed with replayed pretraining data.
- H4: only the highest-quality data, 34B tokens of agent trajectories — mostly Kimi K3 and TML's own models' trajectories, filtered for correctness and alignment.
More from Infra
- ING said to be seller of US$500M data centre loan portfolio — Bloomberg — apples_jimmy · 2026-11-06
- vLLM creators' Inferact now serves 7 trillion tokens a day — woosuk_k · 2026-11-06
- USM turns idle NVIDIA VRAM into Linux swap, ~9 µs vs 55 µs on NVMe — kha333n · 2026-11-06
- Opening a KTV next to Ulanqab's AI data centers is the new 'sure-fire' business, jokes Chinese AI circle — basedjensen · 2026-11-06
- Musk claims SoftBank is defaulting on its turbine orders for AI datacenters — SumitGup · 2026-11-06
- SpaceX plans ~6 GW Arizona datacenter across 9 buildings, no industrial water — elonmusk · 2026-11-06