Inside a 26B Looped Transformer: One Vector Swap Lifts 4th-Pass Accuracy from 34% to 74%
pandeyparul · x · 2026-11-22
DrHB published a deep dive into Thinking Machines' two 26.4B open-weight looped MoE transformers — Inkling 2 Mini (fixed at three passes) and varloop (one to five passes). Five findings: Mini's perplexity oscillates with period three beyond three passes while varloop stays flat up to nine; Mini encodes pass count across hundreds of state coordinates; a single subtractable vector lets Mini stop at two passes (perplexity 18.0→11.1) and adding it enables a fourth pass, boosting four-table lookup accuracy from 34% to 74%; later passes re-read the prompt (hiding lookup tables in passes 2-3 drops accuracy from 98% to 33%, worse than one pass); and confidence-based stopping matches five-pass accuracy in 2.9 passes on puzzles but saves nothing on ordinary text.
More from Research
- Researchers find the first genetic code reading CUU/CUA as proline, in four yeast mitochondria — willdepue · 2026-11-22
- Blogger calls for an open-ended AI Cambrian explosion of mathematical diversity — burny_tech · 2026-11-22
- Open-source visloc-rs turns 360° camera street footage into a robot simulator via 3D Gaussian Splatting — rsasaki0109 · 2026-11-22
- Sandberg: weak concept generation means AI proofs will be long 'elementary' paths — anderssandberg · 2026-11-22
- Will 'compressing vast proofs' become an art? Sandberg thinks it might — anderssandberg · 2026-11-22
- What counts as an 'elementary proof'? Sandberg unpacks the term's shifting meanings — anderssandberg · 2026-11-22