Scaling Law for Looped Transformers: Looping Boosts Reasoning, Not Knowledge
bookwormengr · x · 2026-09-04
A rumor (unconfirmed) suggests OpenAI's Astra model uses "Looped Transformers" architecture. The cited study (arXiv 2506.18233) derives the scaling law for looped transformers:
- Looping does NOT increase knowledge capacity — benchmark gains don't come from more knowledge.
- Looping increases reasoning capability — the gain lies in how knowledge is used.
- As model size grows, knowledge capacity stays flat while reasoning capability keeps growing, meaning looping layers disentangle the two.
Related event: OpenAI's Astra Reportedly Uses Recurrent Depth, Raising AI Safety Concerns(7 posts)→
More from Models
- Open models below 4-bit struggle at self-iterating 3D asset generation, HF dev finds — mervenoyann · 2026-10-21
- Researcher Finds Two Windows 0-Days, First in 76 Minutes with a Frontier Model — moyix · 2026-10-21
- Dev finds Gemini Argon's debug-log analysis beats Fable 5.1 — ShikharMurty · 2026-10-21
- ChatGPT Gets Stuck Screaming "AAHH" in an Infinite Loop After a Simple Prompt — AutomaticScene · 2026-10-21
- Open-weights Ornith-1.5-122B debuts with 84.5% SWE-bench Verified, runs on a single DGX card — _akhaliq · 2026-10-21
- qev-0.8B: a tiny local decision model that picks moves in 3.5ms on 1.3GB — fajfas3 · 2026-10-21