MoE inference study says dropping 28% of routed experts left GSM8K unchanged
dai_app · reddit · 2026-07-25
A small MoE inference study found that the routing tail is surprisingly redundant.
- On a 35B MoE with top-8-of-256 routing, dropping about 28% of the lowest-weight routed experts did not change GSM8K accuracy.
- Baseline was 12/15 correct; every threshold tested, including the most aggressive one, was 13/15.
- 12 of 15 questions produced the exact same final answer across all settings, and response lengths stayed flat.
The setup used greedy decoding, so differences came only from skipping experts. The author notes the sample is tiny and the result is not a general proof, but it suggests the router’s low-weight tail may be a useful target for inference speedups when memory traffic is the bottleneck.
More from Infra
- PyTorch details 2.12x faster on-device streaming ASR on Arm SME2 via ExecuTorch — PyTorch · 2026-10-27
- Revenue per GPU can rise as token prices fall, argues Gavin Baker — GavinSBaker · 2026-10-27
- Anthropic S-1: only $19B total compute spend since inception, overbuild warning — bookwormengr · 2026-10-27
- CoreWeave, You.com and NVIDIA lift BrowseComp 22.9% in 3-week RL post-training sprint — RichardSocher · 2026-10-27
- Grid queues make power architecture an AI-factory decision; Exowatt and Redwood offer fixes — ingliguori · 2026-10-27
- Two CVSS 9.9 Wasmtime flaws allow sandbox escape, one ARM64-only since 2020 — basedjensen · 2026-10-27