Magic details its pretraining recipe: dozens of multiplicative changes and per-generation knowledge evals
magicailabs · x · 2026-09-09
In a follow-up to its pretraining efficiency research, Magic says its recipe is the multiplicative result of tens of changes across architecture, optimizer, training objective, and data — and that fixing minor bugs proved to be a compute multiplier too. To balance data trade-offs, it builds new knowledge evals for each model generation to avoid overfitting over time.
Related event: Magic Claims to Match DeepSeek V4 Pro Pretraining with 1/50 the Compute(11 posts)→
More from Research
- Skyfall AI's SKY-001 world model turns sketches into CAD builds in 3.9s, vs 199s for GPT-6 Astra — rohanpaul_ai · 2026-10-27
- Physicists Let AI Pick Its Own Problem—It Wrote a Publishable Paper in 2 Hours — burny_tech · 2026-10-27
- Researcher flags unresolved data transparency issues in compositionality evals — najoungkim · 2026-10-27
- Separate Forms from Estimators: Why Steer by Difference of Means When You Can Use SGD? — johnhewtt · 2026-10-27
- EMNLP paper probes why some LLM biases survive rewording while others don't — radamihalcea · 2026-10-27
- Datology webinar: Curation Studio matched top data baselines with under 20% compute — josh_wills · 2026-10-27