New arXiv Paper: Router-Aware Importance Sampling Stabilizes MoE RL Training
tokenbender · x · 2026-09-22
tokenbender unpacks the arXiv paper 'Towards Stable and Effective Reinforcement Learning for Mixture-of-Experts' (Furu Wei et al.) plus Xiaomi's ablations:
- RL research mostly targets dense models; RL training instability in MoE architectures is underexplored
- The paper proposes a router-aware approach to optimize importance sampling weights in off-policy RL: rescaling guided by router logits reduces gradient variance and mitigates divergence
- Experiments show significantly improved convergence stability and final MoE performance
- The author flags the router freezing trick in section 5.4 as an intriguing empirical technique
Related event: Xiaomi's MoE RL stability paper: router-aware GRPO tweaks explained(2 posts)→
More from Research
- SGLang Megakernel Pushes Muse Spark to 1,109 tok/s, 2.1x Over Baseline — hsu_byron · 2026-11-12
- ArcGrav: a cross-model Agent Control Language to cut token overhead — ArchitecturalGravity · 2026-11-12
- Open-source dual-arm robot solves tangram puzzles with 90 demos, ~60% success — DanGrover · 2026-11-12
- FutureHouse's deslopper trains Kosmos to write indistinguishably from human experts — proteinrosh · 2026-11-12
- Fleuret amplifies debate: AI can't slash wet-lab experiments in unmapped biology — francoisfleuret · 2026-11-12
- Recursive's Eureka Machine ships Flint: a sparse 40B model with 2.5B active for local agentic coding — agihouse_org · 2026-11-12