Researchers Debate Nesterov's Claim: Inductive Biases May Rescue Neural Network Generalization

DeepMind's Lucas Beyer pushed back on Nesterov's pessimistic claim about learning generalization, suggesting that suitable inductive biases might eliminate the need for gradient-based training. He and NYU's Jonas Geiping debated whether overparameterized networks could instead be trained heuristically.

2026-10-23 ~ 2026-10-23 · 2 related posts

• Lucas Beyer counters Nesterov's generalization pessimism with the transformer inductive-bias argument — giffmana · 2026-10-23
• Researchers debate: can heuristics replace gradient updates in overparameterized nets? — jonasgeiping · 2026-10-23