Top AI researchers are quietly exploring architectures beyond the Transformer
khademinori · x · 2026-11-16
Spending time with top AI researchers reveals many are exploring architectures beyond the standard Transformer: JEPA, diffusion models, Mamba, state-space models and other radically different approaches are advancing in parallel.
The author argues a major architectural breakthrough seems near-inevitable — frontier-level intelligence with significantly less training compute. Total compute consumption will keep growing, but performance per unit of compute should improve dramatically.
More from AGI Musings
- Economist on a world without work: relationships, not jobs, will define us — paulnovosad · 2026-11-16
- Palmer Luckey: a 16-year-old with a laptop will soon outproduce an entire 2015 company — Yamapama · 2026-11-16
- levelsio: Europeans are the most prepared for post-AGI because they're used to not working — alexmacgregor__ · 2026-11-16
- OpenAI Foundation board member: AI accidents will soon become humanity's leading cause of death — ThereWas · 2026-11-16
- Wharton prof Daniel Rock: a wide cultural gulf separates 'everything is compute' believers from everyone else — danielrock · 2026-11-16
- Timothy Lee: AI takeover risk over decades isn't negligible, especially with robotics — binarybits · 2026-11-16