Transformer Limits: AI Industry Undergoes Collective Belief Update
gabriberton · x · 2026-08-06
Gabriel Brostow echoes Chr Szegedy's thoughts on intellectual honesty, admitting the industry broadly underestimated how far the Transformer architecture could scale. He notes a timeline of shifting perspectives: some researchers updated their beliefs in 2020, the majority realized it in 2024, yet some still haven't changed their minds.
More from AGI Musings
- RelianceScope Wins Best Paper: 44% of Student-AI Interactions Are Passive — guzdial · 2026-08-06
- Open Source Models Hit 2016's ASI Standards, Yet the World Remains Unchanged — Dan_Jeffries1 · 2026-08-06
- AGI Won't End Bureaucracy, It Will Reshape It — ryan_t_lowe · 2026-08-06
- Transformer Supply Side Mapped, But Successor Architecture Capabilities Remain Unknown — StewartalsopIII · 2026-08-06
- Are LLMs Getting Worse at Writing? The Rise of Hyper-Dense Prose — Original_Swimming320 · 2026-08-06
- NeurIPS Best Paper Sparks Debate: Are LLMs Converging into an Artificial Hivemind? — gabriberton · 2026-08-06