Qwen3.8 Expert Analysis: 50% Can Be Pruned With Minimal Loss
EyalToledano · x · 2026-08-27
I scored all 24,576 experts in Qwen3.8-Flash-Next using 700k tokens of session traces. Running in 4-bit on an M4 Max, I found that at the median layer, 64 of 512 experts carry half the routing mass, and 256 carry 91%. This implies half the experts are effectively passengers, suggesting we could likely prune 50% of experts while retaining 90%+ accuracy.
Related event: Analysis Finds Half of Qwen3.8 Experts Nearly Idle(2 posts)→
More from Research
- Schmidt Sciences awards $10M to 19 AI humanities research teams — OdedRechavi · 2026-10-15
- Mistral and CERN partner to build a Large Physics Model trained on billions of simulated collisions — MistralAI · 2026-10-15
- Global Transport coupling lifts guided flow generation: ImageNet FID 2.06 to 1.91 — mmbronstein · 2026-10-15
- Your breathing gives you away: Wi-Fi signals can reveal your breath rhythm — TinfoilTricorn · 2026-10-15
- Beyond text correlation: attaching a geometry simulator to ground model reasoning — gerardsans · 2026-10-15
- Sequen CEO Zoe Weil: Ranking Models Should Optimize Purchases, Not Clicks — TWIML AI Podcast · 2026-10-15