AI Scholars Debate: Will Transformers Be Obsolete in 3 Years?
joshua_saxe · x · 2026-08-18
Brief discussion on the future of AI model architecture. Andrew Wilson predicts an architecture that obsoletes the Transformer within three years. Joshua Saxe remains skeptical, noting that current frontier model architectures already seem wild from a GPT-2 perspective, suggesting future progress may involve incremental steps away from classical quadratic transformer blocks rather than a complete replacement.
More from Research
- Stop Indexing at Full Precision: Compressed Vectors Cut Storage by 60x — _reachsumit · 2026-08-18
- RSI Bench Launches to Quantify Recursive Self-Improvement in AI — dhruv2038 · 2026-08-18
- Scale AI launches RSI-Bench: $2,000 per accepted task plus co-authorship for contributors — dhruv2038 · 2026-08-18
- Meta's Dear Algo: Agentic Intent Layer for Unified Search — _reachsumit · 2026-08-18
- Trident: Multi-Aspect Page Annotation for Long-Document VQA — _reachsumit · 2026-08-18
- RAAC Controller Fixes Stagnation in Deep Research Agents — _reachsumit · 2026-08-18