vLLM serves Reflection Beam 501B MoE from day 0, hitting 708 tok/s with DFlash2
woosuk_k · x · 2026-10-17
vLLM announces day-0 support for Reflection's Beam, a 501B-parameter MoE with 23B active parameters per token, optimized on NVIDIA Blackwell. Highlights: coding agents autonomously optimized the model for 2.71× faster batch-1 decode and 1.87× faster prefill; the DFlash2 drafter reaches 708 tokens/s on a single request (TP8) with 3.7 acceptance length; adaptive verification adds 10% throughput at high concurrency. Full docker launch command with fp8 KV cache and FlashInfer backend included. Architecture follows DeepSeek-V3-style MoE routing with custom attention and residual normalization. An AgentX benchmark chart on GB300 shows DFlash2 clearly beating spec-off across concurrency levels.
More from Infra
- Nebius CEO meets Dutch PM as commentator calls it 'the Google of Europe' — robleclerc · 2026-10-17
- SGLang powers Reflection AI's new Beam model in production with day-0 support — ying11231 · 2026-10-17
- speech.cpp v0.8.8 adds Fish Audio drama-3 and M5 Mac speedups, no Python/CUDA needed — solyarisoftware · 2026-10-17
- Joke read: Altman praising the Roadster hints OpenAI may need to rent GWs from SpaceX — matt_slotnick · 2026-10-17
- NVIDIA releases LiLiCorr speculative decoding draft for Nemotron 3.5 Bolt to speed up 74B serving — _akhaliq · 2026-10-17
- Multi-GPU MTP can hurt decode speed: 82 vs 70 tok/s on 4x RTX 3090 — macboller · 2026-10-17