Lucas Beyer counters Nesterov's generalization pessimism with the transformer inductive-bias argument
giffmana · x · 2026-10-23
Google DeepMind researcher Lucas Beyer responds to Nesterov's concise framing on generalization, offering a 'wishful' counterargument: the pessimistic claim is general, but with the right inductive biases and a reasonably organized parameter space — i.e., the transformer architecture — reality could be far less pessimistic.
More from Research
- DeepMind's IPO Paper Gets a 1000x Citation 'Hack' — Thanks to Another Paper Named IPO — _lewtun · 2026-10-23
- Why LLMs dissolve the long-standing barriers to formally verified software — teortaxesTex · 2026-10-23
- Japan's Moonshot IoB Brain Project Publishes 5-Year Research Posters Online — kaixhin · 2026-10-23
- Agent evals should test metric definition drift, argues dev in new post — jylusdev · 2026-10-23
- ruff author asks: how hard is scaling AI chip design from RISC-V to x86? — charliermarsh · 2026-10-23
- Transformer jointly models electrons and magnetism with 0.01% energy error — bravo_abad · 2026-10-23