GPT-2's 'all you really need': the decoder, prompting, and scale
ianand · x · 2026-09-20
ianand summarizes the key moment in our universe: GPT-2 took the decoder half of the Transformer and showed 'all you really need' is — the decoder, a single model for multiple tasks, prompting instead of task fine-tuning (the big unlock), and scale. This line gave rise to today's popular LLMs.
Related event: ianand's Thought Experiment: An AI World Built on the Encoder(9 posts)→
More from Research
- Study: Reflective Data Center Roofs Actually Cool Daytime Ground by 1.8-2.2°C — SumitGup · 2026-11-12
- Good Paper Writing Earns Genuine Praise, Don't Let the Meme Mislead You — Pseudomanifold · 2026-11-12
- HT-Policy Uses Heavy-Tailed Student-t Regression to Close Robot Policy Gap — chris_j_paxton · 2026-11-12
- Google's Physical-Engineering's Last Exam: all 14 frontier models score below 50% — dl_weekly · 2026-11-12
- CypherGuard fixes structurally wrong LLM Cypher queries, hitting 88.5% on CypherBench — JeremyCMorgan · 2026-11-12
- PufferLib 5.0 trains a Rubik's Cube solver at 8.8M SPS on a single 12GB GPU — haydendevs · 2026-11-11