Baidu Open-Sources ERNIE 5.5 Flash Base: 280B-Parameter MoE with Only 14B Active
On November 5, Baidu officially released and open-sourced ERNIE-5.5-Flash-Base, an efficient sparse MoE foundation model: it has 280B total parameters but activates only 14B per token, was pre-trained from scratch on 33.5T tokens, and supports up to 1M token context during fine-tuning and inference. Model weights are now available on Hugging Face, with release notes on ernie.baidu.com and an accompanying official technical blog. The company claims the model achieves SOTA performance among models of the same scale.
Confirmed
- MoE architecture with 280B total parameters and 14B active per token; the backbone has 43 layers, comprising 1 Dense FFN layer and 42 Multi-Router Latent MoE layers (architecture details as relayed by @teortaxesTex)
- Pre-trained from scratch on 33.5T tokens; the official blog states this is the base version released at the end of pre-training, without any mid-training or post-training
- Supports up to 1M token context for both fine-tuning and inference
- Weights are available for download on Hugging Face, along with an official technical blog
- Official benchmarks claim SOTA among same-size models, compared against DeepSeek-V4-Flash-Base, Hy3-pre, and others
- Baidu says the full ERNIE 5.5 Flash will be released later
Why it matters
- The 280B-total/14B-active design balances large capacity with low inference cost, marking another major open-source milestone for the sparse MoE approach
- Million-token context support and open weights give the community new options for fine-tuning and long-document applications
- Interest in architectural details (such as Multi-Router Latent MoE) from outside observers like @teortaxesTex shows the design has sparked discussion in the technical community
2026-11-05 ~ 2026-11-05 · 7 related posts
Primary sources
- Baidu open-sources ERNIE 5.5 Flash: 280B-param MoE with 14B active and 1M context — teortaxesTex · 2026-11-05
- [source] Baidu releases ERNIE-5.5-Flash-Base weights on Hugging Face — External_Mood4719 · 2026-11-05
- Baidu open-sources ERNIE 5.5 Flash Base: 280B params, 14B active, SOTA at its size — BaiduResearch · 2026-11-05
- Baidu releases ERNIE 5.5 Flash Base model on Hugging Face — Xianbao_QIAN · 2026-11-05
- [source] Baidu open-sources ERNIE 5.5 Flash Base: 280B MoE with 14B active params and 1M context — Baidu_Inc · 2026-11-05
- [source] ERNIE 5.5 Flash Base blog: efficiency-first MoE design, pre-training checkpoint open weights — Baidu_Inc · 2026-11-05
1 near-duplicate retellings: BaiduResearch