Inside Beam: 8-of-256 expert routing across 52 layers, sliding window plus NoPE attention
vllm_project · x · 2026-10-16
vLLM details Beam's internals: across 52 layers, each token activates 8 of 256 experts (layers 0-2 use dense MLPs). Attention alternates 512-token sliding-window layers with global NoPE full attention, pairing local processing with long-context access.
Harmony format carries reasoning and tool-use structure, while a separate 5-layer drafter supplies speculative tokens. The 1M context is a configured limit, not evidence of full-window quality.
Related event: vLLM Details Beam Architecture and Kernel-Level Long-Context Optimizations(2 posts)→
More from Infra
- MiniCPM-V 4.7 35B-A3B hits 17+ tokens/s on iPad with ~3GB memory — techNmak · 2026-10-16
- China rolls out 'token factories' and a 2026-28 token economy action plan — nordicinst · 2026-10-16
- Vercel Sandbox Drives go GA: attach TBs of data in sub-second, fork instantly — cramforce · 2026-10-16
- NVIDIA launches RTX Spark for local AI on Windows PCs — ostrisai · 2026-10-16
- SkyRL ships Day-0 RL support for Beam with +62% LoRA decode throughput in vLLM — vllm_project · 2026-10-16
- DFlash2 delivers up to 3.03x decode-only speedup for Beam across five benchmarks — vllm_project · 2026-10-16