The 'sweet lesson' is real: two kinds of ML problems need opposite strategies
mike64_t · x · 2026-10-27
Responding to Rich Sutton's bitter lesson, the author argues a corresponding sweet lesson holds: ML problems fall into two camps. For tasks where you have absolutely no idea how to write the algorithm (e.g. language modeling), the bitter lesson applies—scale wins. For problems where you roughly know the nature of the computation, the sweet lesson says you can restrict how the network operates and prune the search space of possible solutions for better performance.
More from Research
- Physicist lets Claude Opus pick its own research question — result validated by experts — IgorCarron · 2026-10-27
- FlexGlue: sparse-to-dense matching that beats dense accuracy at a fraction of the cost — zhenjun_zhao · 2026-10-27
- Mundo turns a few room photos into an explorable, simulatable 3D world — at 1.5 FPS — mark_k · 2026-10-27
- Mathematics as an experimental science: Marc Lelarge to talk LLM-driven math experiments at ENS — marc_lelarge · 2026-10-27
- DOE Genesis Mission Phase II: AI for particle accelerators and rare isotope separation — prescott · 2026-10-27
- Tenstorrent runs Proteina protein model at high throughput, claims no HBM shortage — gajesh · 2026-10-27