New paper: why trained neural nets develop α≈2 power-law tails in their weights
burny_tech · x · 2026-11-01
A new paper, A Spectral Renormalization-Group View of Learning, argues that neural networks leave a spectral record of learning in their weights. It investigates why well-trained layers commonly develop power-law tails near α ≈ 2, and how understanding this spectral structure can inform better optimizer design and help prevent overfitting. The thread spans 11 parts.
More from Research
- Control Vectors experiment: letting an agent manage its own activations — sloppenheimer · 2026-11-01
- Claude designed scorpion toxin binders: 17 of 211 lab-tested designs bound — shoumikgoswami · 2026-11-01
- Hugging Face open-sources Carbon DNA foundation models with full training recipes — LoubnaBenAllal1 · 2026-11-01
- Carbon-X: midtrained Qwen3-14B beats every open LLM on DNA benchmarks — LoubnaBenAllal1 · 2026-11-01
- RL fails without midtraining: Qwen3-14B stays at chance on FrontierVEP — LoubnaBenAllal1 · 2026-11-01
- FineDNA midtraining lifts DNA scores without degrading English — LoubnaBenAllal1 · 2026-11-01