Albert Gu: Bitter Lesson isn't 'just scale Transformers' — algorithmic innovation matters
sewon__min · x · 2026-11-08
Mamba co-author Albert Gu quips that many have misread Sutton's Bitter Lesson as 'just scaling up transformers.' The lesson is about embracing compute, but innovations in algorithmic primitives can enable far more efficient scaling — echoing Sutton's own note that brute force alone isn't enough and the science of AI is far from finished.
More from AGI Musings
- JEV Gets Its First Paper as Researchers Split on Research Merit vs Closed Business Model — teortaxesTex · 2026-11-08
- "Humans overfit to confirmation too" — a witty mode collapse analogy for AI slop debate — flowersslop · 2026-11-08
- bayeslord: Stop treating AI as a monolith — four distinct goals demand precise language — sebkrier · 2026-11-08
- AI doesn't change how unique your niche interests are, argues dev pixlpa — pixlpa · 2026-11-08
- The 'evolutionary mismatch' chart explaining why humanity will ignore AI warnings — sebpaquet · 2026-11-08
- The meta-question as agent swarms scale: can models beat gradient descent at its own game? — JensHonack · 2026-11-08