Baidu open-sources ERNIE 5.5 Flash Base: 280B MoE with 14B active, 33.5T tokens, 1M context
teortaxesTex · x · 2026-11-06
Baidu's Sun Tianxiang announced ERNIE 5.5 Flash Base, his first release since joining the company: an open-sourced 280B MoE with only 14B active parameters, pretrained on 33.5T tokens, strong on reasoning/coding/long-context benchmarks, and supporting up to 1M context. Commentary highlights the LatentMoE design — multiple sigmoid routers addressing the mismatch between router scores and true expert usefulness without extra activated experts or communication overhead, plus a learnable per-expert scalar in Stable LatentMoE across 43 layers. Post-training is underway.
More from Models
- Unsloth Ships Dynamic V3.1 Quants Claiming 10-15% Accuracy Gains, Reddit Skeptical — fiocalisti · 2026-11-12
- KRAFTON Open-Sources Raon-1 nano: 1.5B Effective Params, 262K Context, On-Device Ready — pmttyji · 2026-11-12
- Robot build test: GPT-6 Astra stabilizes two-board structure, Claude Fable 5.2 falls short — qinzytech · 2026-11-12
- Zvi dissects Anthropic's latest system card, disputing the autonomy threshold call — Don't Worry About the Vase (Zvi) · 2026-11-12
- antirez backs low-bit DeepSeek 4.1 Flash over Qwen 27B dense on Apple M5 Macs — antirez · 2026-11-12
- Gemini re-renders images as text when vision is removed, dropping only 6% WSR — zacharynado · 2026-11-12