GPT-3 to Chinchilla: A Thread Walking Through the Papers That Reshaped AI

thisdudelikesAI · x · 2026-09-12

An ongoing thread recapping landmark AI papers:

GPT-3 (Language Models are Few-Shot Learners, Brown et al., 2020): at 175B parameters, 10x larger than any prior non-sparse model, it showed that scale alone unlocked in-context learning — no fine-tuning or gradient updates, just a few examples in the prompt — sometimes matching fine-tuned SOTA. Few-shot prompting was born here.
Chinchilla (Training Compute-Optimal LLMs, Hoffmann et al., 2022): corrected Kaplan's scaling laws, showing most large models were badly undertrained; the rule of thumb became 20 tokens per parameter, reframing how the industry spends compute.

A useful primer list of the field's foundational papers.

More from Models