Richard Socher: Transformer's real breakthrough was killing recurrence, not attention
DeryaTR_ · x · 2026-10-27
Recursive CEO Richard Socher argues the biggest insight of the Transformer paper wasn't inventing attention — it was eliminating recurrent layers entirely. He uses this to explain why most people, including top scientists, underestimate AI: they underrate the combinatorial space of existing ideas across physics, chemistry, biology and CS, and predicts superintelligence will soon drive nearly all new discoveries and inventions.
More from AGI Musings
- Late Interaction: frontier AI solving millennium problems while being dumb at 1000 things makes AGI look farther — lateinteraction · 2026-10-27
- OpenAI claims rigorous Yang-Mills existence proof; physicist Woit calls the announcement misleadingly vague — burny_tech · 2026-10-27
- OpenAI internal model claims to refute ETH; Impagliazzo, Paturi and Williams to organize independent review — aminkarbasi · 2026-10-27
- MIRI researchers publicly press Anthropic: breakneck RSI racing contradicts its governance record — geoffreyirving · 2026-10-27
- Is the virtual cell the next AlphaFold? A debate on solvability — burny_tech · 2026-10-27
- Automation won't accelerate science: measurement and interpretation are the bottleneck — fkasummer · 2026-10-27