Sleep-like pruning: dynamic sparse networks beat dense ones with under 10% of synapses
dlevenstein · x · 2026-11-15
dlevenstein describes work where a network alternates learning with sleep-like pruning and regrowth. This "dynamic sparse training" produces networks that outperform dense networks with <10% of their synapses, develop cortex-like wiring motifs, and handle multi-task learning better. He also notes his students Viggy and Huihong are applying to PhD programs, and he'll join the "Careers in NeuroAI" panel at the NIH BRAIN Initiative professional development workshop to discuss predictive learning, sleep, and neuroAI.
Related event: Dynamic Sparse Training Beats Dense Nets With Under 10% of Synapses(2 posts)→
More from AGI Musings
- Protesters rally at San Francisco "Stop the AI race" demo with Don't Build Skynet signs — ronbodkin · 2026-11-15
- Open source only 3-6 months behind frontier AI, talk sparks doubts over frontier labs' returns — SchmidhuberAI · 2026-11-15
- 1938 Newspaper Clip Sparks Debate: Tech Panics Mask a Long March From Embodied Experience — sebpaquet · 2026-11-15
- Google's new AIS team unveils DuoDiscover with 4.09x kernel speedups and LEAP formalizing Weil conjectures — VioletNPeng · 2026-11-15
- Oought-Backed Epistemics Fund Offers $500k–$5M Grants for Truth-Seeking Tools — NathanpmYoung · 2026-11-15
- Researcher now runs AI detector Pangram on papers before deciding how carefully to read — TuhinChakr · 2026-11-15