New theory work proves 'good weights' are everywhere in large neural networks
yuxiangw_cs · x · 2026-11-04
Researcher yuxiangwcs discusses Sunil Madhow's latest theoretical work (Astra can 'speak' Qwen weights): good, generalizing weights are proven to be everywhere in large neural networks — even the dumbest optimizer generalizes adaptively.
Illustrative example: if a small network implements your perfect classifier but you fit a much larger one, a gazillion weight configs achieve zero training error, some overfitting wildly. Yet randomly drawing one among them just works. The figure sketches the geometry of interpolating parameters, deep lifts of the witness, and high-risk parameter regions.
More from Research
- Zachary Huang's 1-hour deep learning crash course praised as the perfect refresher — ZacharyHuang12 · 2026-11-04
- Biologist spends 10+ years developing a gene therapy to cure his own cystic fibrosis — NikoMcCarty · 2026-11-04
- New paper: exponential residual flows bring probability-flow ODEs to discrete diffusion — ArnaudDoucet1 · 2026-11-04
- Researchers Push Back: Frontier Labs Are Overlooking Chemistry in AI for Science — anshulkundaje · 2026-11-04
- Coding agents build baselines for 36 top-journal AI methods, matching ~95% of scores — owl_posting · 2026-11-04
- EGA beats PCA at recovering dimensions from LLM item embeddings, study finds — GolinoHudson · 2026-11-04