MoE routing expansion experiment: Qwen3.6-35B hits 84.34% on GPQA-Diamond
Specific-Tax-6700 · reddit · 2026-09-13
What happened
vagrillo, author of the moe-expansion branch in llama.cpp, ran a full GPQA-Diamond (198 questions) comparison of MoE routing expansion vs native routing and a dense model:
- Qwen3.6-35B-A3B + expanded routing (Q80): 84.34% (167/198)
- Qwen3.8-27B dense (Q80): 83.84% (166/198)
- Qwen3.6-35B-A3B, native routing: 81.82% (162/198)
Setup was controlled: greedy decoding, 32K budget, identical prompts, one RTX 5000 PRO 48GB, single run per config.
Why the author isn't convinced
- 35B expanded vs 27B dense differs by just 1 question — parity, not a win
- Expanded vs native is 12 wins / 7 losses (net +5), not significant at n=198
- 10 of the 12 expansion wins are Chemistry-only; Physics is saturated, Biology flat
- One run, one seed
Questions for the community
- Which benchmarks pair well with GPQA-Diamond for routing comparisons
- How many runs are needed for a paired comparison
- Is a Chemistry-only effect plausible or a red flag
- Has anyone else observed a real MoE expansion effect on Qwen MoE models
More from Models
- Anthropic files for Nasdaq IPO: $16.3B H1 revenue, compute spend down to 0.5x revenue — DeepLearningAI · 2026-10-31
- ThursdAI crew debates: with Opus 7 coming, who still cares about better benchmarks? — thursdai_pod · 2026-10-31
- Swift 1.5 Flash Next Heretic quants add IQ4_XS and IQ3_S tuned for reverse engineering — jjusko20 · 2026-10-31
- 14 models tested on 960 analyst tasks: AI scores 97/100 as analyst, picks winners no better than random — HeyAmit_ · 2026-10-31
- Anthropic adds invisible text watermark to Claude outputs starting Nov 3 — petesena · 2026-10-31
- GPT-6 Luna vs GPT-6.1 Terra vs GPT-6.1 Sol vs GPT-6 Astra in Pixel Art — Angaisb_ · 2026-10-31