Reflection's 156-page Beam report: swapping RMSNorm kernels yields 10% speedup
brandondamos · x · 2026-11-07
- Reflection released a 156-page tech report on its Beam model covering pre-training, mid-training, SFT, RL and OPD, including infra, data and scientific decisions.
- Brandon Damos (NeMo AutoModel) highlighted practical training-infra details from it:
- TE's RMSNorm kernel was hogging SMs and starving NCCL; swapping in cuDNN's RMSNorm gave a 10% speedup.
- They disable Python's automatic GC and have all ranks collect together every few steps — same as NeMo AutoModel.
- An FA4 bug at head dim 256 forced them to use 128 instead.
- Fusing all metric all-reduces into one made logging cheaper at scale.
- The report is praised for its honest, practical detail.
Related event: Reflection AI Open-Sources Beam with 156-Page Technical Report(8 posts)→
More from Infra
- AI server makers Dell, HPE, Lenovo and Supermicro hike prices by up to 45%, even on signed deals — churchkey · 2026-11-22
- Where will the world store AI's exploding data? Oceans, space, DNA and glass — ChuckDBrooks · 2026-11-22
- Codex Code Mode calls get 10ms faster by reusing tool catalogs across cells — charliermarsh · 2026-11-22
- AI ecosystem guide 2026 maps 120 tools across 18 categories from training to agents — blaizedsouza · 2026-11-22
- Google AI members get free monthly Cloud credits: $10/mo for Pro, up to $100/mo for Ultra 20x — lxfater · 2026-11-22
- Inkling 2 mini's 1M-context KV cache dissected: only 8 of 48 layer visits keep full cache — teortaxesTex · 2026-11-22