vLLM's built-in DSpark delivers up to 2.23x output throughput gain
vllm_project · x · 2026-10-29
vLLM ships built-in DSpark support with measured output throughput gains of 1.92–2.16x in FP8 and 2.00–2.23x in NVFP4 versus the matching baseline (TP8, 200 real prompts, greedy decoding, ≤512 output tokens). The release also adds low/medium/high reasoning effort (default high), multi-image prompts for Pixtral vision, and day-0 RL training support for Mistral Large 4 in SkyRL with vLLM rollouts.
More from Infra
- Evo 2 genomic model now runs on vLLM: 7B fits 1-megabase context on a single H100 — anshulkundaje · 2026-10-30
- Meta's GenRecs Tokenization lifts ad conversions +0.35% with semantic vocabulary — Meta_Engineers · 2026-10-30
- Parallel Language Models paper: multi-token generation at AR quality, up to 5.7x speedup — aaron_lou · 2026-10-30
- Oracle investor day: 1GW AI site earns ~$80B contract value at ~28% ROI, 3.6-year payback — rohanpaul_ai · 2026-10-30
- Sam Altman: Local inference is a terrible allocation of scarce global compute — morqon · 2026-10-30
- AWS blueprint: deploy NVIDIA Nemotron 3.5 Super VL (120B MoE) on EKS with Dynamo and vLLM — AWS ML Blog · 2026-10-30