Stepfun Open-Sources Step 5 Preview with 600B MoE and 1M Context

Stepfun has open-sourced Step 5 Preview, a 600B-parameter sparse MoE model with 27B activated per token, native multimodal support, and 1M-token context, featuring Sparse GQA attention that cuts indexer overhead by 8x for long-horizon agent tasks.

2026-10-15 ~ 2026-10-15 · 2 related posts

• StepFun open-sources Step 5 Preview: 600B-param MoE, 27B active, 1M context — teortaxesTex · 2026-10-15
• Step 5 Preview architecture blog details Sparse GQA with block-wise token merging cutting indexer cost 8x — teortaxesTex · 2026-10-15