Reverse-engineering Inkling 2 Mini: one vector direction controls how many loop passes run

naftalinus · reddit · 2026-11-23

A blogger probed inside Inkling 2 Mini (a 26B looped transformer) and found its middle 12 layers run three times per token, with the pass count encoded in one direction of the inter-pass vector. Adding that vector enables a working 4th pass (4-table lookup puzzles: 34% → 74%); subtracting it stops after 2 passes (perplexity 18.0 → 11.1). Without intervention, Mini only performs well at 3, 6, and 9 passes — at 4 passes perplexity degrades from 10.5 to 15.2. Full write-up with interactive figures is available.

Original post →

More from Research

Research channel →