vLLM announces Day-0 support for Step 5 Preview across Blackwell and Hopper
vllm_project · x · 2026-10-15
- vLLM announced Day-0 support for Step 5 Preview, covering sparse attention, MoE, multi-token prediction and vision, with serving work spanning both Blackwell and Hopper GPUs.
- The engineering spans the sparse indexer, FP8 MegaMoE and cache transfer, with a thread detailing how the pieces fit together plus kernel results and validation.
More from Infra
- Inworld opens up its consumer-AI inference API, citing tens of trillions of tokens a month — thetripathi58 · 2026-10-15
- When will 16GB machines run local LLMs on par with Opus-class models? — bit_freak · 2026-10-15
- vLLM P/D prototype hits 91.24% prefix cache hit rate on 12× GB300 — vllm_project · 2026-10-15
- vLLM ships Day-0 support for Step 5 Preview, cuts sparse prefill latency 2.2% on B300 — vllm_project · 2026-10-15
- vLLM details Step 5 Preview's region-based sparse attention: score summaries, then attend — vllm_project · 2026-10-15
- UW's COUNT stores n-gram counts on host memory, giving 1B LMs the equivalent of 57% more compute — UW · 2026-10-15