vLLM adds Elastic Expert Parallelism: grow/shrink MoE GPU pools under live traffic
PyTorch · x · 2026-09-26
PyTorch announced Elastic Expert Parallelism in vLLM, which lets operators add or remove GPUs from an active Mixture-of-Experts deployment during live traffic with minimal serving interruption and downtime.
NVIDIA's Itay Alroy will present 'Elastic Expert Parallelism in vLLM' at PyTorch Conference North America 2026, covering the architecture, key implementation details, open challenges, and roadmap—including what happens when EP size changes and how NIXL EP enables grow/shrink under live traffic.
More from Infra
- Running Qwen2.5-Coder-3B on Android: Jupyter + Pydantic Evals on-device eval pipeline — alex_b3b · 2026-11-11
- 22% of Nvidia revenue now books to Singapore, but only ~1% is local buyers — Gregory_C_Allen · 2026-11-11
- Huawei's Kirin 9050 NPU claims offline 30B MoE on smartphones, +140% perf — AIFlow_ML · 2026-11-11
- Stratechery: Intel vs AMD's diverging earnings and Intel's need for an old-new path — Stratechery · 2026-11-11
- a16z Podcast: Mike Volpi on What the 1990s Internet Boom Teaches Us About the AI Buildout — a16z Podcast · 2026-11-11
- Alibaba files to list chip-design unit T-Head in Hong Kong to fund AI push — pstAsiatech · 2026-11-11