Reverse-engineering Inkling 2 Mini: one vector direction controls how many loop passes run
naftalinus · reddit · 2026-11-23
A blogger probed inside Inkling 2 Mini (a 26B looped transformer) and found its middle 12 layers run three times per token, with the pass count encoded in one direction of the inter-pass vector. Adding that vector enables a working 4th pass (4-table lookup puzzles: 34% → 74%); subtracting it stops after 2 passes (perplexity 18.0 → 11.1). Without intervention, Mini only performs well at 3, 6, and 9 passes — at 4 passes perplexity degrades from 10.5 to 15.2. Full write-up with interactive figures is available.
More from Research
- Base log perplexity is an extremely strong predictor of Epoch and OpenAI ECI, researchers note — stochasticchasm · 2026-11-23
- ECHO-style SFT losses in RL dodge the elicitation-only criticism, researcher argues — stochasticchasm · 2026-11-23
- Idea: humans write high-level proofs, AI fills in lemma details in Lean — theshawwn · 2026-11-23
- Claude points to Lean blueprints, Lamport structured proofs, and Gödel's speedup theorem — theshawwn · 2026-11-23
- Two-person lab GSI claims 250+ breakthroughs across 8 fields with custom AI-built solvers, not brute-force compute — udmrzn · 2026-11-23
- Geoffrey Irving: statistical physics offers a route to obfuscated arguments in scalable oversight — AndrewCritchPhD · 2026-11-23