Baidu open-sources ERNIE 5.5 Flash Base: 280B MoE with 14B active params and 1M context
Baidu_Inc · x · 2026-11-05
Baidu released ERNIE-5.5-Flash-Base, an open-weight sparse MoE base model: 280B total parameters with 14B activated per token, pre-trained from scratch on 33.5T tokens, and supporting up to 1M-token context for fine-tuning and inference. FP8 and BF16 weights are available.
Key points:
- The weights are the checkpoint at the end of pre-training, with no mid- or post-training, positioned as a common starting point for domain adaptation, instruction tuning and RL research.
- Architecture and training system were co-designed for efficiency, balancing activated compute, KV cache and long-context cost across attention, residual connections, MoE layers and the output head. The blog and architecture diagram show Latent-Gated Sparse Attention, Multi-Router Latent MoE (Top-10 of 512 experts), ECHO, MultiMax Head and an R2L tokenizer.
- Benchmark charts (same eval pipeline vs DeepSeek-V4-Flash-Base, Hy3-preview-Base, MiMo-V2.5-Base) show it leading most tasks, e.g. MMLU-Pro 78.4, RepoBench 54.6, RepoQA 91.7.
Weights are on Hugging Face; details on the ERNIE blog.
More from Infra
- Baseten launches Grid: AI gateway with intelligent routing to cut coding agent costs — baseten · 2026-11-05
- Microsoft rejects multibillion-dollar AI deal with UK's Nscale, FT reports — nordicinst · 2026-11-05
- Microsoft Details Running Coding AI Locally on 128GB RTX Spark Windows PCs Without Token Fees — lee_stott · 2026-11-05
- On 64GB combined memory, is there anything better than Qwen 3.8 27B yet? — xdcfret1 · 2026-11-05
- Revolut self-hosts GLM on 56 B200s, cutting coding token cost to ~$0.30/M — alex_verem · 2026-11-05
- Dev: skip any GPU under 256GB for local AI, prices are about to fall — AIFlow_ML · 2026-11-05