Looped transformers explained: Inkling 2 Mini loops 12 middle layers 3x for deeper reasoning without more weights
Prompt Engineering · youtube · 2026-11-22
A long-form video explainer of looped transformers, using Thinking Machines' newly open-sourced Inkling 2 Mini as the running example.
How it works
- The model has 24 layers, with the middle 12 run three times per token — 48 layer steps without storing more weights.
- The idea traces to Universal Transformers (2018); related work includes ByteDance Ouro and NVIDIA Dory.
- It's MoE-based, runs on a laptop, and is available via Ollama (Mac / DGX Spark).
Pass dial
- 1–5 loop passes: more passes reach the same reasoning with shorter thinking.
- Looping improves reasoning depth but not one specific thing; the cost trades off memory vs compute.
Why it worried people: moving reasoning inside the loop undermines chain-of-thought monitoring; Goodfire published interpretability research on the model.
All benchmarks shown are Thinking Machines' own, measured on their research version.
More from Models
- Claude concedes a scalar-bound mistake after reading the author's blog post, keeping its curvature arguments — bremen79 · 2026-11-22
- Pixel art benchmark: Gemini 4 Neon wins on quality but costs 13x more and 4x slower than GPT-6.2 Sol — Angaisb_ · 2026-11-22
- Uncensored Inkling 2 Mini v2: self-distillation lifts useful answers from 52% to 79% with open pipeline — Turbulent-Sky5396 · 2026-11-22
- GPT-5.4 with open-source KYNEX harness outscored standalone GPT-6 Astra on ExtractBench: 92.97 vs 91.9 F1 — the-random-guy-2002 · 2026-11-22
- Researcher claims Gemini Neon helped him land a potential Nature-worthy physics idea — xennygrimmato_ · 2026-11-22
- Unnecessary Hyphens and Semicolons Are the New En-Dash in Claude Writing — bttyeo · 2026-11-22