ERNIE 5.5 Flash Base blog: efficiency-first MoE design, pre-training checkpoint open weights
Baidu_Inc · x · 2026-11-05
Following the release of ERNIE-5.5-Flash-Base, Baidu published the official blog and Hugging Face links. The 280B-total/14B-active sparse MoE was pre-trained from scratch on 33.5T tokens, and the released weights are the end-of-pre-training checkpoint with no mid- or post-training, meant as a common starting point for domain adaptation, instruction tuning and RL research.
The blog explains the efficiency-first design: later training stages sample repeatedly and agentic apps add multi-turn, long-context costs, so the model balances activated compute, KV cache and long-context overhead across components; the pre-training corpus prioritizes capabilities that are hard to add afterwards, such as repository-scale code and long-context understanding.
More from Infra
- oMLX lets a second Mac boost prefill up to 2.9x on big-memory Macs — pcuenq · 2026-11-05
- FineWeb-30M speedrun record falls to 14.2s on one 8xH100 node, a 1.64x speedup — RobertTLange · 2026-11-05
- HydraDB: a Rust graph database that keeps state in S3 and treats query workers as disposable compute — JeremyCMorgan · 2026-11-05
- OpenSSH 10.6p2 released with a workaround for a critical vulnerability — Animux · 2026-11-05
- Microsoft rejects multibillion-dollar AI deal with UK's Nscale, FT reports — nordicinst · 2026-11-05
- Microsoft Details Running Coding AI Locally on 128GB RTX Spark Windows PCs Without Token Fees — lee_stott · 2026-11-05