Baidu open-sources ERNIE 5.5 Flash-Base: 280B-param MoE with 14B active, built for long-context agents

机器之心 · wechat · 2026-11-05

Baidu released ERNIE 5.5 Flash-Base with open weights, a sparse MoE base model targeting long-running agent workloads: 280B total / 14B active params, pretrained from scratch on 33.5T tokens with 128K training context extended to 1M at inference.

Key architecture choices:

Hybrid sparse attention: only 6 of 43 layers keep full token-level access via MLA with top-k sparse attention; the other 37 use heavily compressed HCA (128 tokens per cache entry), slashing KV cache costs.
Multi-Router MoE: multiple sigmoid routers vote instead of a single top-k scorer, with 512 routed experts (10 active) plus 1 shared expert at no extra inference cost.
ECHO objective: instead of bolt-on MTP modules, the trunk's final layer directly learns multi-token prediction, doubling as a built-in speculative-decoding draft model with emergent extrapolation.

Baidu reports strong code gains over DeepSeek-V4-Flash-Base (HumanEval+ 82.3 vs 69.5). The open base weights let developers run their own domain adaptation, fine-tuning and RL.

Original post →

Baidu open-sources ERNIE 5.5 Flash-Base: 280B-param MoE with 14B active, built for long-context agents

More from Infra