Meta's Loop Scaling Laws: sparsity and recurrence yield ~2-3x efficiency gains in MoE models
fly51fly · x · 2026-10-02
Meta AI researchers introduce Loop Scaling Laws, the first scaling law jointly modeling recurrence and MoE sparsity alongside model size and data. Key findings:
- A bounded, sparsity-conditional recurrence mapping characterizes effective-parameter gains from looping
- Predicts held-out loss more accurately than prior laws, recovering dense and MoE laws as special cases
- Sparsity delivers 3x active-parameter efficiency; recurrence yields 2x total-parameter efficiency on reasoning; combined scaling advances the frontier further
- At trillion-token scale, a looped MoE with law-derived recurrence matches a 2x larger non-looped MoE on reasoning benchmarks at matched compute, and enables test-time scaling via recurrence
Related event: Meta Unveils First Scaling Laws for Looped Mixture of Experts(3 posts)→
More from Research
- Mobile AI Factory: agents auto-build on-device SLMs with 19x less GPU time — pmttyji · 2026-11-16
- Berkeley paper: LLMs answer with stale values even when the update is in context — ombre-ai · 2026-11-16
- Open-source TotalSegmentator spots a 42mm dilated aorta on a routine chest CT — MaziyarPanahi · 2026-11-16
- Tivadar Danka Publishes 4,000-Word Visual Guide to Graph Theory for Machine Learning — TivadarDanka · 2026-11-16
- METR time-horizon chart updated: doubling every 1.7 months since Opus 4.5, 'Model 4' ~38h — ben_j_todd · 2026-11-16
- GTAgentClinic: one-line install, plug in Claude Code or Codex for a 3D hospital shift — Michael_D_Moor · 2026-11-16