vLLM ships Day-0 support for StepFun's Step-5 Preview: 604B MoE with 1M context
vllm_project · x · 2026-10-15
The vLLM project announced Day-0 support for StepFun's Step-5 Preview, a 604B multimodal MoE with hybrid sparse/sliding-window attention, a 1M-token context window, and multi-token prediction.
Key points:
- Official Docker image (vllm/vllm-openai:step-5-preview) and recipe support both FP8 and NVFP4 checkpoints.
- FP8 runs single-node on 4×B300 with TP4/TEP4/DEP4 layouts (MegaMoE on Blackwell); H200 needs 8 GPUs. NVFP4 also runs on 4×B300.
- Day-0 RL training via verl: GRPO + LoRA on FSDP2 with vLLM rollouts, including R3 routing replay and adapter-only weight sync (PR still draft).
- A P/D disaggregation prototype on 12×GB300 (DEP4 prefill + DEP8 decode, FP8/NIXL/384 GiB cache store) hit 91.24% prefix-hit rate; two 1,000-request OCR runs scored 90.4% and 89.4% with zero errors (pre-release test).
Related event: vLLM Ships Day-0 Support for Step 5 Preview Across Blackwell and Hopper(7 posts)→
More from Infra
- Moving 1PB out of the cloud costs up to $74K in egress fees alone, $500K all-in — DavidLinthicum · 2026-10-15
- Robotics data gap: just 300K hours of manipulation data globally vs 100M+ needed, $3B spend projected — 0xSammy · 2026-10-15
- Ramp raises $1.85B at $60B valuation, says 85,000 firms save $1B+/month with AI tools — SethGRosenberg · 2026-10-15
- AI's smartest bear case is supply: $500B GPUs idle, but demand is elastic — bittingthembits · 2026-10-15
- Intel said to have secured foundry commitments from Broadcom and Google, Apple wins on 14A — BenBajarin · 2026-10-15
- Magnitude open-source inference engine beats Ollama by over 2x with auto-tuned kernels — nickbaumann_ · 2026-10-15