Reflection AI open-sources Beam 1.0 tech report: 23.8T tokens on 6,144 GB300s
RobertTLange · x · 2026-11-07
Reflection AI released a 156-page technical report for Beam 1.0, detailing the full training recipe:
- Pretraining: 23.8T tokens (13.7T unique × 1.73 epochs) on 6,144 GB300s for 21.9 days at 21% MFU
- Optimizer: AdamC instead of AdamW/Muon, weight decay ∝ LR² → 0, peak LR from a fitted law η ∝ N^−α·D^−0.32·B^0.5
- Async RL: no staleness bound (avg 5 policy versions behind), 100M+ rollouts on ≤10.5K GB300s
- Agentic + alignment experts merged via full-vocab on-policy distillation
- Stability: depth-scaled residuals (1/√n), σ-gated attention; busiest MoE expert ends at 1.04× uniform
The 52-layer MoE has 23.8B active / 49B total params with 256 top-8 experts and gated QK attention. Weights are open-sourced under Apache-2.0, with comparisons against Nemotron and Olmo 3 training setups.
Related event: Reflection AI Open-Sources Beam with 156-Page Technical Report(8 posts)→
More from Models
- Researcher claims Gemini Neon helped him land a potential Nature-worthy physics idea — xennygrimmato_ · 2026-11-22
- Unnecessary Hyphens and Semicolons Are the New En-Dash in Claude Writing — bttyeo · 2026-11-22
- Backend check shows "GPT-6.2 Astra" traffic actually routed to GPT-6 Astra entirely — bdsqlsz · 2026-11-22
- Testing "Gemini 4 Neon" in Cursor: blogger teases unconfirmed new Google model — IndraVahan · 2026-11-22
- Yacine: Neon Is Literally Better Than Anything Anthropic Has — yacineMTB · 2026-11-22
- Gemini hides a periodic table easter egg: tap model button 7 times — lipeng0820 · 2026-11-22