A parallel universe where AI innovation went to the Transformer encoder instead
ianand · x · 2026-09-20
A counterfactual AI history discussion: someone imagined a parallel universe with no GPT-2 moment, where all technical and product innovation poured into the encoder half of the Transformer instead. Ian Brownless replied that in our world, AI is generative — more inventive and open-ended.
It's essentially a thought experiment about what LLMs would look like if the autoregressive decoder paradigm hadn't won.
Related event: ianand's Thought Experiment: An AI World Built on the Encoder(9 posts)→
More from AGI Musings
- Fleuret: if AI designs drugs as fast as it makes dumb videos, regulation will buckle — francoisfleuret · 2026-11-12
- Newsweek op-ed: US should fund hospitals and universities in federated AI learning — Saul_Loveman · 2026-11-12
- 10-line deterministic polynomial-time primality test relies on OpenAI's recent quasi-GRH result, making AKS obsolete — fortnow · 2026-11-11
- AI2027 Author Kokotajlo Shifts AGI Median Timeline to Around 2030 — JacquesThibs · 2026-11-11
- Ron Bodkin: automated research taste advances faster than expected, but the evidence is weak — ronbodkin · 2026-11-11
- Should devs still hand-write code? Author proposes 1,000-hour baseline before AI agents — bendee983 · 2026-11-11