Meta's Loop Scaling Laws: Sparsity Gives ~3x Active-Param Efficiency, Recurrence ~2x on Reasoning
facebook · hf · 2026-10-01
Meta introduces Loop Scaling Laws, the first to jointly model recurrence and MoE sparsity alongside model size and data, via a bounded sparsity-conditional recurrence mapping. The laws predict looped-model loss better than prior alternatives and recover dense/MoE laws as special cases. Empirically: sparsity yields 3x active-parameter efficiency, recurrence 2x total-parameter efficiency on reasoning; at trillion-token scale, a law-derived looped MoE matches a 2x larger non-looped MoE at matched compute.
Related event: Meta Unveils First Scaling Laws for Looped Mixture of Experts(3 posts)→
More from Infra
- tandem.cpp v0.3.0: 2-6x faster prefill for models too big for VRAM, Maya DeepSeek-V4.1 support — smflx · 2026-11-16
- Developers poll real costs of running coding agents on remote VMs for work — radim11 · 2026-11-16
- Datadog's new gateway returns 402 when a model's budget runs out—so agents just retry — YvesMulkers · 2026-11-16
- Max-quality local Minimax H3 video on a 4090: community-tested int8 setup — newaccount47 · 2026-11-16
- Local V2V outfit transfer on a 16GB M3 MacBook Air: 15.7 min cut to 2.4 min — might_thinker · 2026-11-16
- FlashStates hits 3.14x vLLM prefill throughput for hybrid-architecture agentic inference — 机器之心 · 2026-11-16