Astra's rumored looped transformer gets a technical debunk, with Oriol Vinyals citing Universal Transformer
OriolVinyalsML · x · 2026-09-03
The Information reported that OpenAI's rumored Astra model uses a "recurrent depth or looped transformer" architecture, sparking debate.
Sebastian Raschka published a detailed technical breakdown to contextualize and debunk the hype around looped transformers, noting he shared a related architectural analysis two months ago.
DeepMind VP Oriol Vinyals replied "Universal Transformer FTW", tagging author Mostafa Dehghani — implying the "new" approach echoes the 2018 Universal Transformer.
Related event: Debate Erupts Over OpenAI Astra's Rumored Looped Transformer Architecture(3 posts)→
More from Models
- Meta's Muse Spark 1.3 is now free on OpenCode — ramagetime · 2026-09-03
- DeepLoop paper makes looped transformers scalable; rumor claims frontier models are 48 layers looped twice — StartupYou · 2026-09-03
- Leaked: OpenAI's Secret 'Bel' Model Slated for Year-End After Astra's Agent-1 Stage — haider1 · 2026-09-03
- Reddit user on Gemini 3.8 Flash: more effort, similar result — Correct_Tomato1871 · 2026-09-03
- Qwen3.8-Flash-Next on 2x3090 + DDR4: expert cache PR lifts decode from 17 to 25-29 t/s — Extension-Bid-639 · 2026-09-03
- IBM releases Granite 4.2: free open-source models built for AI agents, runs locally — krvarshney · 2026-09-03