Inside Beam: 501B params, only 23B active per token via 8-of-256 expert routing
vllm_project · x · 2026-10-16
A vLLM thread breaks down Reflection Beam 1.0: 52 layers (3 dense, then MoE) with 256 experts and 8 active per token—23B of 501B parameters active. Attention alternates a 512-token sliding window with global NoPE attention; 1M configured context (officially noted as a config limit, not a quality claim). Harmony handles reasoning, final responses and tool use, with a separate 5-layer drafter supplying speculative tokens.
More from Infra
- Neocloud stocks sink on Gartner's "80% will fail" report; analyst calls it garbage — McDonaghMatthew · 2026-10-16
- JapanFold Claims One Tenstorrent Server Beats an Nvidia DGX B200 on Bio Models at a Third of the Price — DavidBennett__ · 2026-10-16
- vLLM ships Day-0 RL support for Beam in SkyRL, +62% LoRA decode throughput — vllm_project · 2026-10-16
- DFlash2 speeds Beam decode up to 3.03× across benchmarks, vLLM reports — vllm_project · 2026-10-16
- CoreWeave founders cashed out $3.3 billion before the stock's meltdown — SumitGup · 2026-10-16
- The Verge spent a week with Nvidia's RTX Spark Arm laptops: here's what we learned — tomwarren · 2026-10-16