Goodfire probes Inkling 2 Mini: looped model layers read from "future" layers
fatihdin4en · x · 2026-11-21
After Thinking Machines released Inkling 2 Mini, a 26B looped transformer, interpretability firm Goodfire got early weight access and studied how information flows through blocks and loops. Counterintuitively, looped layers can write to themselves or read from "future" layers; decomposing per-layer inputs reveals which layers, blocks, and loops communicate. Goodfire argues our ability to understand transformers can evolve alongside their architectures.
More from Models
- Thinking Machines Lab ships Inkling 2 Mini: looped MoE with 3B active params for local setups — ariG23498 · 2026-11-21
- Model "personality" is the second axis, and Google's data curation is winning it — iruletheworldmo · 2026-11-21
- Inkling 2 Mini runs locally in 16GB RAM: 5.4B-active MoE tops its size class — simonguozirui · 2026-11-21
- Mech interp hackathon derailed as everyone freaks out over Claude solving Riemann hypothesis — burny_tech · 2026-11-21
- Ollama CEO in Fortune: enterprise demand for open models keeps growing — ycombinator · 2026-11-21
- vLLM day-0 support for looped Inkling: 2.68x faster decode on GB200 with 78% fewer kernels — vllm_project · 2026-11-21