GPT-6 Astra reportedly uses loop transformers, adding compute depth instead of parameters
pranavmarla · x · 2026-09-16
A cited discussion around SemiAnalysis claims GPT-6 Astra (unconfirmed) uses loop transformers: instead of growing parameter count, the model passes through its layers multiple times, adding compute depth while keeping weights fixed.
- If depth beats size, the FLOP-counting arms race changes shape — inference cost curves look very different when weights are reused.
- The poster reads labs' less aggressive parameter scaling roadmaps as a signal that their own research shows diminishing returns from raw parameter growth.
- Note: this is industry rumor/hearsay, not officially confirmed.
Related event: Report: GPT-6 Astra Said to Use Loop Transformer Architecture(3 posts)→
More from Infra
- Transformers.js 4.3 ships structured output, new model architectures, and Safari WebGPU support — nicodotdev · 2026-09-16
- Novita open-sources Chord W4A16 MoE kernels, up to 2.15x faster Kimi K2.x inference on B300 — vllm_project · 2026-09-16
- VCs say the GPU shortage is really a capital access problem: 30% down payment prints money — sarahdrinkwater · 2026-09-16
- Developer builds Linux subsystem for Haiku with OCI runtime and container support — unixterminal · 2026-09-16
- Anthropic signs lease at Queensland data center park costing ~$30B, online 2027 — DigitalColmer · 2026-09-16
- Transformers models now run natively in vLLM with no port required — pcuenq · 2026-09-16