Why GPT-6's rumored recurrent-depth architecture could change inference economics
panic_in_the_galaxy · reddit · 2026-09-24
A detailed breakdown of the rumor that GPT-6 uses recurrent depth — reusing parts of the Transformer stack multiple times instead of a single pass through unique layers — decoupling effective depth from parameter count.
Key points:
- Combined with MoE, recurrence yields two scaling axes: which computation (expert routing) and how much (recursion depth); different passes route through different experts
- Looped-MoE research shows gains even with matched parameters, FLOPs and KV-cache budgets — not just hidden extra compute
- More reasoning could happen in the hidden state (as in Huginn) rather than via thousands of chain-of-thought tokens; test-time compute becomes internally allocated
Caveat: OpenAI hasn't published the architecture; recurrent depth is unconfirmed reporting, and per-token adaptive depth is extrapolation from Geiping et al. 2025 and Bae et al. 2025 (Mixture-of-Recursions).
More from Models
- Xiaomi ships open-weights MiMo-V2.6-Pro, tops open-source AI index at 46, rivaling Claude Opus 5 — ycombinator · 2026-09-24
- Friends in rural Sweden who only knew ChatGPT are now asking about Meta's Muse — gabriel1 · 2026-09-24
- User Teases That OpenAI's Post-Training Team Is Cooking an 'Opus 4-5' Rival — sloppenheimer · 2026-09-24
- Claude Opus 5.5 tops VoxelBench, GPT-6 Sol ranks third — legit_api · 2026-09-24
- GPT 6 computer use impresses: drives user through 15 banking sites in one go — altryne · 2026-09-24
- IKEA Assembly Benchmark: Top Model Score Jumped From 28% to 80% in 10 Months — emollick · 2026-09-24