Baidu Open-Sources ERNIE 5.5 Flash Base: 280B Params, 5% Active, 1M Context
xiaohu · x · 2026-11-05
Baidu released ERNIE 5.5 Flash Base, an open-source foundation model for developers to fine-tune into domain-specific models.
- Trained from scratch with 280B total parameters, only 14B (5%) activated at inference, supporting 1M-token context.
- KV-cache layers are split: 37 of 43 layers read compressed context for global info, while 6 layers can return to the original text for precise detail retrieval.
- MoE routing picks 10 of 512 experts via multi-router scoring; multi-token prediction reuses the final layer instead of an extra module, which also drafts candidates for verification at inference.
- Benchmarks (see companion post) show same-size SOTA versus DeepSeek-V4-Flash-Base and others.
More from Models
- Unsloth Ships Dynamic V3.1 Quants Claiming 10-15% Accuracy Gains, Reddit Skeptical — fiocalisti · 2026-11-12
- KRAFTON Open-Sources Raon-1 nano: 1.5B Effective Params, 262K Context, On-Device Ready — pmttyji · 2026-11-12
- Robot build test: GPT-6 Astra stabilizes two-board structure, Claude Fable 5.2 falls short — qinzytech · 2026-11-12
- Zvi dissects Anthropic's latest system card, disputing the autonomy threshold call — Don't Worry About the Vase (Zvi) · 2026-11-12
- antirez backs low-bit DeepSeek 4.1 Flash over Qwen 27B dense on Apple M5 Macs — antirez · 2026-11-12
- Gemini re-renders images as text when vision is removed, dropping only 6% WSR — zacharynado · 2026-11-12