Hobbyist Converts Qwen3.5-2B Dense to MoE, With Full Benchmarks
Bayliner1980 · reddit · 2026-11-20
Independent researcher bayliner1980 released a dense-to-MoE conversion of Qwen3.5-2B (1.3B active), the largest yet, with weights and GGUF on Hugging Face.
Key methodology improvements:
- "Dead experts" turned out to be merely unused—adding math, science, and multilingual data revived them and fixed some code limitations;
- Faster conversion;
- A new fact-recall metric quantifies how much factual knowledge survives conversion.
Results: Wikitext-2 ppl 13.60→14.68 (+7.9%), C4 ppl +15.0%; task mean 0.526→0.487; fact recall on 287 facts dropped 0.948→0.760. The author notes factual-knowledge loss remains the main open problem.
More from Models
- Hy4 Released: 770B MoE, 49B Active, 1M Context — EmetInteractive · 2026-11-20
- Follow-up to the Gemini 4 lineage leak thread — wavefnx · 2026-11-20
- Leaked Gemini 4 lineage: neon scales hydrogen's base, beats carbon at lower cost — haider1 · 2026-11-20
- Pro subscriber asks for a $50 mid tier between Pro and Max to access Fable — jikesar968 · 2026-11-20
- Claude models 82% off on RouterHub: Opus 5.5 at $0.72 in, Fable 5.2 at $1.43 in — zebird0 · 2026-11-20
- Fan-made chart claims MAI Code 2 matches 96% of Claude Opus 5.5 coding perf at 5% of the cost — marlene_zw · 2026-11-20