"We had the Transformer paper before LLMs — nothing even close since," dev argues

dosco · x · 2026-11-06

Developer dosco argues that "we had the transformer paper before LLMs and nothing even close since then" — a one-line distillation of a widespread anxiety among practitioners that mainstream LLMs remain Transformer variants while attention alternatives like linear architectures have yet to displace it. A good conversation starter on whether architectural innovation has hit a plateau.

Original post →

"We had the Transformer paper before LLMs — nothing even close since," dev argues

More from AGI Musings