The bitter lesson isn't 'naive wins' — it's whatever scales best with compute
pzakin · x · 2026-11-09
- A common misreading: "the naive approach always wins in the end." Wrong — the bitter lesson says the approach that scales most effectively with compute wins.
- Analogy: among functions scaling linearly, exponentially, factorially, only the growth rate with x (compute) matters — not initial size or reliance on human knowledge.
- It's not anti-complexity: NLP is complex, and LLMs are hilariously complex too; they win because they scale hardest.
More from AGI Musings
- AI has broken science's main evaluation proxy, argues computational biologist — anshulkundaje · 2026-11-09
- Two major cardiology trials fail, a reality check for 'AI will cure all disease' — anshulkundaje · 2026-11-09
- Toby Ord: a small share of impossible tasks would crush the 80% time horizon — tobyordoxford · 2026-11-09
- Learning is compression: a long-form essay on manifold hypothesis and why intelligence is possible at all — burny_tech · 2026-11-09
- 'AI democratizes Jevons paradox itself': efficiency fuels even more usage — cocktailpeanut · 2026-11-09
- Toby Ord: share of impossible tasks could skew METR's AI time horizon — tobyordoxford · 2026-11-09