Paper Shows Even Random Sampling Finds Generalizing Weights in ReLU Networks

A new arXiv paper by Yu-Xiang Wang and Sunil Madhow proves that generalizable "good weights" are ubiquitous in large ReLU networks, meaning even the dumbest optimizer doing random sampling can find them, offering a new theory for why deep learning generalizes.

2026-11-04 ~ 2026-11-04 · 2 related posts

• "Deep lifts" paper: ReLU networks adaptively generalize even with the dumbest random optimizer — yuxiangw_cs · 2026-11-04
• New theory work proves 'good weights' are everywhere in large neural networks — yuxiangw_cs · 2026-11-04