Baidu open-sources ERNIE 5.5 Flash Base: 280B MoE with 14B active params and 1M context

Baidu_Inc · x · 2026-11-05

Baidu has released and open-sourced ERNIE 5.5 Flash Base, the first release by tianxiangsun since joining the company.

Key facts:

Architecture: sparse MoE with 280B total parameters, only 14B activated per token
Training: pre-trained from scratch on 33.5T tokens with up to 128K training context; supports 1M-token context for fine-tuning and inference
Positioning: weights are the raw pre-training checkpoint (no mid/post-training), meant as a common starting point for domain adaptation, instruction tuning, and RL research
Efficiency focus: every component (attention, residual connections, MoE layers, output head) was redesigned to balance activated compute, KV cache, and long-context cost; the corpus emphasizes capabilities hard to add later, like repo-scale code and long-context understanding

Post-training is already underway, with more releases teased.

More from Companies & People