Top AI researchers are quietly exploring architectures beyond the Transformer

khademinori · x · 2026-11-16

Spending time with top AI researchers reveals many are exploring architectures beyond the standard Transformer: JEPA, diffusion models, Mamba, state-space models and other radically different approaches are advancing in parallel.

The author argues a major architectural breakthrough seems near-inevitable — frontier-level intelligence with significantly less training compute. Total compute consumption will keep growing, but performance per unit of compute should improve dramatically.

Original post →

More from AGI Musings

AGI Musings channel →