MoE training failure details: router collapsed without freezing, multimodal experts suspected
stochasticchasm · x · 2026-09-22
The author asks when 'MOPD' happened, noting only mixRL has been observed publicly. He calls the training logs rough: even by R3, the router died without being frozen, and he wonders whether multimodality caused it, given how awkwardly experts can behave in multimodal models.
Related event: async 4 discloses data mix, MoE router crash sparks discussion(3 posts)→
More from Research
- Andrew Ng, ICML 2027 Program Chair, asks for ideas to fight AI slop and submission explosion — xuanalogue · 2026-09-22
- SteerDuplex: full-duplex speech model gains 44.5pp in steerability, new SteerBench released — ScaleAI · 2026-09-22
- Sarah Hooker suspects fake AI paper submissions cluster at a few universities — sarahookr · 2026-09-22
- Xiaomi MiMo's CodeMIDAS scales agentic coding RL environments from code itself — _akhaliq · 2026-09-22
- Rollout scheduling: the underappreciated infra trick boosting inference efficiency — stochasticchasm · 2026-09-22
- Conference fixes for AI paper flood: per-author submission caps and faster fake-profile detection — sarahookr · 2026-09-22