vLLM adds an experimental plugin to split attention and FFN serving
vllm_project · x · 2026-07-24
vLLM introduced an experimental AFD Plugin that implements Attention-FFN disaggregation for MoE serving.
- Attention and expert/FFN workloads have different scaling needs, so the plugin splits them into separate services instead of forcing one shared topology.
- The design keeps the same vLLM serving surface and avoids a fork.
- It targets both NVIDIA GPUs and Ascend NPUs.
The attached architecture diagram shows entry points for OpenAI-compatible APIs and vLLM serve, plus separate attention, AFD connector, and FFN roles.
Related event: vLLM Introduces Experimental AFD Plugin to Boost MoE Inference(2 posts)→
More from Infra
- Robot builder says Bob died after a battery mod fried its Raspberry Pi brain — chrismatthieu · 2026-07-24
- Baseten adds GLM-5.2 fine-tuning and production deployment in Loops SDK — baseten · 2026-07-24
- Vercel lets Workflows pin state to a region, with AI Gateway limits coming soon — cramforce · 2026-07-24
- 30 open-source AI repos covering agents, local inference, and workflow automation — Shruti_0810 · 2026-07-24
- A 1-bit Bonsai 27B quant runs locally on a 16GB MacBook Air — fuckAIbruhIhateCorps · 2026-07-24
- SpaceX reportedly turns away Falcon 9 customers beyond 2028 as it bets on Starship — BasedRaddka · 2026-07-24