vLLM serves Beam at 1,769 tok/s on 4x B300 NVFP4, publishes Docker image and recipes
vllm_project · x · 2026-10-16
vLLM shares Beam serving resources: Docker image vllm/vllm-openai:reflection-beam, plus official recipes and blog.
In chat tests at concurrency 16, 4×B300 NVFP4 TP4 delivered 1,769 output tok/s and 8×B300 FP8 TP8 delivered 1,300 (285 and 217 tok/s at concurrency 1). Configs use different GPU counts and precisions — not an equal-resource comparison; BF16 KV with variable output lengths. The recipe also lists memory needs: NVFP4 294GB on disk / 353GB GPU minimum, FP8 503/604GB, BF16 1,003/1,204GB.
Related event: SkyRL and vLLM Deliver Day-0 Training and Serving Support for Beam(2 posts)→
More from Infra
- MiniCPM-V 4.7 35B-A3B hits 17+ tokens/s on iPad with ~3GB memory — techNmak · 2026-10-16
- China rolls out 'token factories' and a 2026-28 token economy action plan — nordicinst · 2026-10-16
- Vercel Sandbox Drives go GA: attach TBs of data in sub-second, fork instantly — cramforce · 2026-10-16
- NVIDIA launches RTX Spark for local AI on Windows PCs — ostrisai · 2026-10-16
- SkyRL ships Day-0 RL support for Beam with +62% LoRA decode throughput in vLLM — vllm_project · 2026-10-16
- DFlash2 delivers up to 3.03x decode-only speedup for Beam across five benchmarks — vllm_project · 2026-10-16