Dense-to-MoE conversion: +32% GPU speed but perplexity degrades ~40%
Bayliner1980 · reddit · 2026-10-13
An update on converting dense models to MoE: bayliner1980 released Qwen2.5-1.5B-MoE-A0.9B plus GGUF versions of both 0.5B and 1.5B conversions.
Results: WikiText-2 perplexity rises 41% (10.41→14.63), zero-shot mean accuracy drops 0.565→0.462 with ARC hit hardest; but generation speed gains +32% on GPU (RTX 4060) and +41% on CPU — the first conversion clearly faster than dense even on GPU. Honest caveat: at this size quality matches the faster Qwen2.5-0.5B, so it's a proof of concept; the approach should pay off more on larger models where MLP dominates compute. GGUF is larger than dense due to untied output head in llama.cpp MoE.
More from Research
- RTNav: open-source modular asynchronous agent for real-time zero-shot object navigation — rsasaki0109 · 2026-10-13
- AI-CFD benchmark culture: a mesh-free solver takes a cow to Mach 2 — burhop · 2026-10-13
- The Implicit Curriculum Hypothesis wins COLM outstanding paper award, powered by NSF public compute — IanArawjo · 2026-10-13
- Computational biology shifts as AI builds 'better predictors' in under a day — jmschreiber91 · 2026-10-13
- Model weights are only 1 of 4 VRAM allocations: why loaded LLMs still OOM — blaizedsouza · 2026-10-13
- 30,000 hours of egocentric video: world model learns hands, not object outcomes — xuanalogue · 2026-10-13