Betting on transformers: incremental improvements will beat radically new LLM architectures
max_paperclips · x · 2026-11-09
The author frames the architecture debate as an efficiency question: pure RNN-based LLMs might have eventually reached GPT-6-tier capability, but would have been so inefficient that nobody would bother. His prediction: the transformer as a class is here to stay, with future breakthroughs improving that design — even hybrid RNN-like components are still essentially transformers — rather than something radically new.
More from AGI Musings
- 'You Can Get Arbitrarily Smart Without Reading Books' — But Does Anyone Have an Example? — alexisgallagher · 2026-11-09
- Future CEO of agentic orgs will be a system designer, not a McKinsey alum — GregCook2011 · 2026-11-09
- Box CEO: agent swarms turn AI productivity gains from 20-50% into unbounded — inductionheads · 2026-11-09
- When humans become the bottleneck, lift the loop: System 1 steers, agents deliver — dotey · 2026-11-09
- Full video: Cathie Wood on why ARK rejects AI doom and predicts a labor shortage — rohanpaul_ai · 2026-11-09
- Cathie Wood blames Anthropic's CEO for AI doom headlines, predicts labor shortage instead of job losses — rohanpaul_ai · 2026-11-09