Hobbyist Converts Qwen3.5-2B Dense to MoE, With Full Benchmarks

Bayliner1980 · reddit · 2026-11-20

Independent researcher bayliner1980 released a dense-to-MoE conversion of Qwen3.5-2B (1.3B active), the largest yet, with weights and GGUF on Hugging Face.

Key methodology improvements:

Results: Wikitext-2 ppl 13.60→14.68 (+7.9%), C4 ppl +15.0%; task mean 0.526→0.487; fact recall on 287 facts dropped 0.948→0.760. The author notes factual-knowledge loss remains the main open problem.

Original post →

More from Models

Models channel →