ERNIE 5.5 Flash Base: 280B-A14B built for 1M context, tops 25 base-model benchmarks
karminski3 · x · 2026-11-06
Baidu released ERNIE-5.5-Flash-Base, a 280B-total/14B-active model pretrained on 33.5T tokens, designed from scratch for 1M ultra-long context and extreme sampling efficiency in post-training/agent loops. Its raw base weights already top 25 of 30 benchmarks in a unified pipeline (MMLU-Pro 78.4, +9.2 over V4-Flash; RepoQA 91.7). Key techniques: a 6:1 mix of 37 HCA layers (128× compression) and 6 MLA-SA layers (per-token dynamic sparsity) with low-rank gating saving 75% of gate params; Multi-Router scoring plus 2048-dim half-width Latent MoE (512 experts, top-10); and ECHO self-recursion replacing the external MTP module — the trunk's last layer does multi-token prediction itself with only 0.36B extra params, trained with dynamic depth sampling. The accompanying charts also compare context strategies across ERNIE 5.5, DeepSeek-V4-Flash, V4.1-Flash and Qwen3.8-Flash-Next.
More from Infra
- 40 of 41 Data Center Ballot Measures in 2026 Midterms Went Against AI Builders — Acceptable-Offer-518 · 2026-11-12
- Azure Day at KubeCon: AKS Runs 22,655 Agents on 65 Nodes in Live Demo — davemccollough · 2026-11-12
- Gas-for-AI-datacenters forum co-located with CoRL shows energy demand spillover — carlosdponx · 2026-11-12
- a16z podcast: Mike Volpi on Cisco's 70+ acquisition playbook and the AI buildout — a16z · 2026-11-12
- AMD to pass Intel in server revenue by mid-2027 as share gains 6 points a year — BenBajarin · 2026-11-12
- Study: Reflective Data Center Roofs Actually Cool Daytime Ground by 1.8-2.2°C — SumitGup · 2026-11-12