Training-free MoE router tweak cuts Qwen 35B reasoning tokens by 8.5%
Specific-Tax-6700 · reddit · 2026-09-04
A new paper describes a training-free inference-time MoE optimization: expanding the expert budget (A3B → A4B+) only in late transformer layers with linear decay. On full MMLU-Pro (714 questions, Qwen3.6-35B-A3B) it cuts mean reasoning tokens 8.5% and latency 10.9% while accuracy stays statistically unchanged (84.5% vs 84.0%). Dubbed "Succinct Convergence," the method needs zero training; paper, beta code and full JSON results are on Zenodo/GitHub.
More from Research
- acs infra launches free base model API offering Llama 3.1 8B and 405B for LLM psychology researchers — amplifiedamp · 2026-10-21
- ACS Research Launches Free Platform Giving Researchers Access to Llama 3.1 Base Models — jankulveit · 2026-10-21
- LessWrong Essay Introduces LM Psychology: Just Talking to an AI Still Beats Fancy Evals — jankulveit · 2026-10-21
- LiFT loops a shared DiT core to beat dense baseline by 3.34 FID with 60% fewer params — burny_tech · 2026-10-21
- Stanford researcher clashes over genomics foundation model's cherry-picked variant prediction benchmarks — anshulkundaje · 2026-10-21
- Gemini Agents Crack 59-Year-Old Fusion Math Conjecture in Hours — DynamicWebPaige · 2026-10-21