Baidu open-sources ERNIE-5.5-Flash-Base: 280B MoE with 14B active params and 1M context
kimmonismus · x · 2026-11-06
Baidu released ERNIE-5.5-Flash-Base with open weights: a 280B MoE model with only 14B active parameters per token, pretrained from scratch on 33.5T tokens. It's a pure base model with no mid- or post-training, intended for labs to adapt to domain tasks, reasoning, and agents.
Key efficiency tricks:
- Only 6 of 43 layers keep token-level full-context access; the other 37 compress every 128 tokens into a single cached entry, keeping the KV cache small while still supporting up to 1M-token context for fine-tuning and inference.
- Multi-Router MoE: several routers jointly pick experts without extra activated-expert compute or communication cost.
- ECHO: multi-token prediction reuses the model's own final layer, no separate MTP module.
Preliminary same-pipeline benchmarks against DeepSeek-V4-Flash-Base, Hy3-preview-Base and MiMo-V2.5-Base show ERNIE-5.5-Flash-Base leading on many exam, code and long-context benchmarks, including MMLU 93.3, HumanEval+ 82.3, BBH CoT 89.2 and RepoQA 91.7.
More from Infra
- Data center NIMBYism is a corporate good-neighbor problem, says ex-Kraft vet — BecauseCulture · 2026-11-06
- Used AMD MI100 home-lab test: a 3D-printed shroud cools 300W at just 2,400 rpm — Much-Farmer-2752 · 2026-11-06
- Transcribing 11.3 hours of Nixon tapes in under 3 minutes with Nemotron 3.5 for ~$0.30 — vanstriendaniel · 2026-11-06
- Intel down 46% in 3 months: the bull case rests on 18A costs down 40% and $10B extra data center revenue — tengyanAI · 2026-11-06
- AI bots hit a site 5x more than humans: 1.23M requests, ChatGPT needs ~100 fetches per visitor — marclou · 2026-11-06
- Qwen3.8-27B as local coding agent burns 16k tokens thinking, never calls tools — previaegg · 2026-11-06