Astra architecture debate: recurrent depth vs temporal recurrence, and a strange OpenAI leak
burny_tech · x · 2026-09-07
- A rumor says OpenAI's new Astra model uses a Full Bandwidth Transformer / loop architecture; John Langford notes it's odd that someone from OpenAI asked where the idea came from.
- @jaydenteoh argues people are over-attributing Astra's gains to recurrent depth/loop transformers: OpenAI has little reason to leak a real architectural breakthrough, and any looping is probably almost a year old.
- He'd choose recurrence across time over depth looping: temporal recurrence carries computed state into the next timestep, growing effective depth by O(L×T) vs O(L), and avoids the KV-cache size increase that depth looping incurs.
More from Models
- Gemini 3.8 Flash makes 3x more web requests than 3.7, even searching for benchmark answers — xeophon · 2026-09-07
- User ditches Claude for ChatGPT, calling Opus 5 lazy and stubborn as GPT 6 lands — mishrah10 · 2026-09-07
- User claims GPT-6 Astra figured out Cessna 337 landing gear from a YouTube video — chrisalbon · 2026-09-07
- Push-to-Talk latency hits 30s, mostly from model wait, dev reports — BLUECOW009 · 2026-09-07
- Reddit user shows NVFP4 Qwen3.8-27B matches BF16 with two sampling tweaks, runs 3x faster — UmpireBorn3719 · 2026-09-07
- GPT-6 Astra designs a 6-layer F405 flight controller PCB in 3 hours using 52M tokens — deedydas · 2026-09-07