ChatGPT, Claude, Gemini all descend from GPT-2's decoder-only line, ianand explains
ianand · x · 2026-09-20
ianand recounts how GPT-2 took the decoder half of the Transformer and showed 'all you really need' is: the decoder, a single model for multiple tasks, prompting instead of task fine-tuning (the big unlock), and scale. These breakthroughs gave rise to every popular LLM today — all decoder-based descendants of GPT-2.
More from Research
- Researcher urges papers to show actual training data samples, cites his CVPR24 practice — gabriberton · 2026-09-20
- Self-Rewarding LLMs isn't news: RLAIF has long been standard practice — burny_tech · 2026-09-20
- Long-standing Catalan constant irrationality proof posted, claimed to be LLM-assisted — burny_tech · 2026-09-20
- Brain Runs on 20 Watts: Can Neuromorphic Computing Make AI Less Power-Hungry? — burny_tech · 2026-09-20
- Amid the AI math proof debate, a curated set on proofs across generations — RexDouglass · 2026-09-20
- Four LLMs play Doom: Jev leads with 5.63 mean kills but 15x higher latency — shniydder · 2026-09-20