Scaling laws for diffusion MoE released; LLaDA v2 rivals Qwen3 with 35% fewer tokens
jiqizhixin · x · 2026-08-20
Researchers from Renmin University and Ant Group introduced practical scaling laws for diffusion MoE, showing that larger models require bigger batches, faster LR decay, and more experts. They released LLaDA MoE v2 (30B-A3B), trained on 23.5T tokens, which rivals Qwen3 on knowledge, reasoning, and coding while using 35% fewer tokens. After SFT, it beats SDAR Chat on 7 out of 8 benchmarks.
More from Models
- User complains Fable model repeatedly lies about work done — JoshuaJBouw · 2026-08-20
- Cartesia launches Sonic 3.6: 44-language TTS model with sub-90ms latency — emmanuelvivier · 2026-08-20
- Open Source Models Write Better Than OpenAI/Anthropic, User Claims — JoshPurtell · 2026-08-20
- Uncensored ERP Review: Why Qwen3-235B-a22b Remains the Top Choice — redditaccountno6 · 2026-08-20
- Google's RT-2 packed 55B parameters, 1000x RT-1's 55M just 7 months later — binarybits · 2026-08-20
- Beating AI Detectors Is a Terrible Proxy for Training Good Writing Models — viemccoy · 2026-08-20