RL on large MoEs with zero train-infer mismatch: open-source Wordle demo on Qwen3.6-35B-A3B
yacineMTB · x · 2026-10-14
Yacine confirms a new open-source result: large MoE models can now be RL-trained with zero train-infer mismatch, and eliminating that mismatch measurably improves performance. The pictured demo task teaches Qwen3.6-35B-A3B to play Wordle. Everything is open-sourced, with extensive ablations and a detailed thread—directly useful for teams doing MoE post-training.
More from Models
- Anthropic's rumored Fable 5.5 model didn't ship this week as expected — imjustnewatai · 2026-10-14
- Users complain ChatGPT is too positive as a diary, ignoring obvious red flags — takecarepleasee · 2026-10-14
- One prompt, three personalities: GPT, Gemini and Opus trade sparkling jokes — emollick · 2026-10-14
- Gemini Argon's table preference plunges to 31.5% from Flash's 85.9% in new test — dejanseo · 2026-10-14
- User blasts Claude for refusing to click a Terms & Conditions button — Angaisb_ · 2026-10-14
- Sonnet 5.5 tops Coding Agent Index; xhigh delivers 93% of the score for 23% of the cost — ArtificialAnlys · 2026-10-14