The Bitter Lesson Was Misread: More Compute Makes Your Algorithm Matter More
tw_killian · x · 2026-11-10
A lengthy X post argues the "Bitter Lesson" has been misinterpreted by a generation of scientists: the brief period when scaling transformers did most of the legwork reward-hacked the field's thinking.
Key points:
- The more compute you pour into training, the MORE important your algorithm is—some algorithms scale, some don't.
- Lots of energy went into directions that turned out to be dead ends, while whole dimensions of scaling remain unexplored.
- The quoter summarizes: learn RL ("that good Bellman stuff"), implying RL-style methods are the promising scaling direction.
More from AGI Musings
- Fable's 5-year advice for applied AI careers: build skill at the constraint — brandon_galang · 2026-11-10
- Ben Goertzel predicts continued massive AI industry growth through 2027 — bengoertzel · 2026-11-10
- Ex-AI Security Lead: AI Luminaries' Cyber Takes Are Wrong, Real Expertise Will Reclaim Authority — joshua_saxe · 2026-11-10
- SlopOps: mistaking agentic coding capability for competitive advantage is playing the slop lottery — generativist · 2026-11-10
- Eric Schwitzgebel's 22-minute interview on the possibility of AI consciousness — eschwitz · 2026-11-10
- Yudkowsky: AI fiction lags code badly — 'we may only be alive because gradient descent generalizes policies so poorly' — sebkrier · 2026-11-10