Thinking Machines details training Inkling 2 Small with native 4-bit forward passes
clarejtbirch · x · 2026-11-14
Thinking Machines published a new Connectionism blog post detailing how it trained Inkling 2 Small with a native 4-bit forward pass from the very first step of pretraining through the final RL rollout. The core insight: training with fewer bits is faster, but quantization error can interfere with learning — their approach keeps 4-bit precision stable across the entire training pipeline. A rare public deep-dive into low-bit training methodology from a frontier lab, shared widely as "finally a blog that doesn't need new models."
Related event: Thinking Machines Details End-to-End 4-Bit Training of Inkling 2 Small(10 posts)→
More from Models
- rasbt maps Inkling 2 Small's 594B architecture built on attention residuals — rasbt · 2026-11-14
- Sokoban Eval: New 66-Room Benchmark Makes LLMs Play Classic Puzzle Game — banteg · 2026-11-14
- Lab post-training researchers flood-likes post mocking on-policy distillation reality gap — teortaxesTex · 2026-11-14
- Mitsuhiko: Anthropic has no image model, yet its code that paints is shockingly good — mitsuhiko · 2026-11-14
- Rapid Gemini releases at the frontier spark super-exponential AGI optimism — iruletheworldmo · 2026-11-14
- Google exec confirms Gemini 4 pre-training run has started, its most ambitious yet — Independent-Wind4462 · 2026-11-14