vLLM Upgrades EPD Disaggregation for Multimodal Serving: Lower TTFT, Higher Per-GPU Throughput
vllm_project · x · 2026-11-11
vLLM announced major upgrades to its Encoder/Prefill/Decode (EPD) disaggregation architecture for multimodal serving. Since encoder load varies independently of prefill/decode demand with media size and modality, EPD splits stages into independently deployed vLLM instances.
Key changes:
- Metadata-only requests let downstream instances consume remote embeddings without re-processing media; DeepSeek V4.1 Flash metadata-only EPD is on main
- Dedicated encoder execution: separate Model Runner V2 path, encoder-only CUDA Graph support, reduced checkpoint-loading I/O
- Remote embedding transfer via NIXL and Mooncake integrations
- Cross-encoder output reuse through Mooncake Store
- Lower orchestration overhead integrated with vLLM Router, SMG, and llm-d
Core capabilities, including MiMo and XPU support, ship in vLLM v0.32.0.
Related event: vLLM's EPD Disaggregation Cuts Multimodal TTFT by Up to 47%(2 posts)→
More from Infra
- Stratechery: Intel vs AMD's diverging earnings and Intel's need for an old-new path — Stratechery · 2026-11-11
- Alibaba files to list chip-design unit T-Head in Hong Kong to fund AI push — pstAsiatech · 2026-11-11
- AWS CEO expects to be short on compute 'for the next couple of years' as demand outstrips supply — firstadopter · 2026-11-11
- Natural language search over autonomous driving data with Cosmos-Embed1 + OpenSearch — 4310sy · 2026-11-11
- Fully local MiniMax H3 Image + LTX-2.5 pipeline generates a shot in under 5 minutes on one RTX 5090 — Hailuo_AI · 2026-11-11
- Dev claims ~700 tokens/s running Qwen 3 27B on a single RTX 5090 — IgorCarron · 2026-11-11