Magic details its pretraining recipe: dozens of multiplicative changes and per-generation knowledge evals
magicailabs · x · 2026-09-09
In a follow-up to its pretraining efficiency research, Magic says its recipe is the multiplicative result of tens of changes across architecture, optimizer, training objective, and data — and that fixing minor bugs proved to be a compute multiplier too. To balance data trade-offs, it builds new knowledge evals for each model generation to avoid overfitting over time.
More from Research
- Lila Sciences cofounder Ben Kompa makes MIT TR35 list for autonomous science push — tw_killian · 2026-09-09
- 100 rule-based verifiers and 300 tasks: team builds a working RL recipe for video models — DanielKhashabi · 2026-09-09
- Researchers' 2022 paper warned privacy issues like this were coming — thegautamkamath · 2026-09-09
- Local Bad Behavior In RL Doesn't Produce Emergent Misalignment, Per Hacker-Opus — 1a3orn · 2026-09-09
- Speculative thread: capability-RL-trained LLMs may learn a 'use every affordance' heuristic — 1a3orn · 2026-09-09
- Pretraining progress is mostly from data: 12x compute gains from data vs 3.7x from models — sebkrier · 2026-09-09