Dynamic Sparse Training Beats Dense Networks With Under 10% of Synapses
dlevenstein · x · 2026-11-15
A tweet highlighting research on dynamic sparse training, which alternates learning with sleep-like pruning and regrowth. The resulting networks outperform dense networks using less than 10% of their synapses, develop cortex-like wiring motifs, and do better at multi-task learning.
Related event: Dynamic Sparse Training Beats Dense Nets With Under 10% of Synapses(2 posts)→
More from Research
- ROS 2 Record-and-Replay for the SO-101 Arm Using MCAP and Foxglove — kamathsblog · 2026-11-15
- Google's new AIS team unveils DuoDiscover with 4.09x kernel speedups and LEAP formalizing Weil conjectures — VioletNPeng · 2026-11-15
- Compositional reasoning under RL post-training paper accepted to NeurIPS 2026 — yifeiwang77 · 2026-11-15
- How EmbeddingGemma 2 is trained: staged Gemini distillation down to 128-dim Matryoshka embeddings — gabriberton · 2026-11-15
- EmbeddingGemma 2 topped HuggingFace trending for a week, a spot usually held by LLMs — gabriberton · 2026-11-15
- SJTU's Value-as-Memory Gives Robot Policies Task-Progress Memory, +29.4pp on Real Tasks — jiqizhixin · 2026-11-15