ChatGPT, Claude, Gemini all descend from GPT-2's decoder-only line, ianand explains
ianand · x · 2026-09-20
ianand recounts how GPT-2 took the decoder half of the Transformer and showed 'all you really need' is: the decoder, a single model for multiple tasks, prompting instead of task fine-tuning (the big unlock), and scale. These breakthroughs gave rise to every popular LLM today — all decoder-based descendants of GPT-2.
Related event: ianand's Thought Experiment: An AI World Built on the Encoder(9 posts)→
More from Research
- Google's Physical-Engineering's Last Exam: all 14 frontier models score below 50% — dl_weekly · 2026-11-12
- CypherGuard fixes structurally wrong LLM Cypher queries, hitting 88.5% on CypherBench — JeremyCMorgan · 2026-11-12
- PufferLib 5.0 trains a Rubik's Cube solver at 8.8M SPS on a single 12GB GPU — haydendevs · 2026-11-11
- 10-line deterministic polynomial-time primality test relies on OpenAI's recent quasi-GRH result, making AKS obsolete — fortnow · 2026-11-11
- WSP Weight Edits Shown Equivalent to Constrained Activation Steering on Qwen — kastnerkyle · 2026-11-11
- codec.audio benchmarks neural audio codecs on noisy in-the-wild speech; MOSS tops — kastnerkyle · 2026-11-11