The Bitter Lesson Misread: Sutton Himself Doesn't Buy the LLM Path

dotey · x · 2026-11-10

The "correction" is just rereading the original: The Bitter Lesson already argued for general methods like search and learning that scale with compute; treating that as a fresh insight means most people only read the title.
"Wrong" vs "right" cleverness is a spectrum, not a chasm: CNNs hard-coded human priors like translation invariance and locality; with enough data, ViTs dropped those priors and won— itself a bitter lesson. AlphaGo→AlphaZero→MuZero kept stripping out human knowledge.
"Scales with compute" is a retrospective criterion: Every researcher believes their clever idea is the right kind; expert systems thought so too. If the lesson reduces to "good innovations are good, bad ones are bad," it's neither bitter nor useful.
Sutton himself doesn't buy LLMs: In his Dwarkesh interview he said plainly that LLMs imitate human data, lack goals, and don't learn from experience—the path itself misses the Bitter Lesson. His "clever new methods" mean continual learning and learning from interaction, which LLMs lack. Reading him as endorsing Transformer-style innovation turns a radical critique into palatable chicken soup.

More from AGI Musings