Magic claims >10x more efficient pretraining, matching DeepSeek V4 Base for ~$0.5M
oran_ge · x · 2026-11-15
AI startup Magic published a research update on compute-efficient pretraining, claiming its recipe is now >10x more compute-efficient than leading open-weight base models.
- Matches DeepSeek V4 Pro Base with 50x fewer FLOPs—roughly half of GPT-3's training compute, $0.5M on GB200
- Scaling 10x further ($4M) meaningfully outperforms all publicly available open base models on perplexity evals
- Per its scaling laws, matching that capability with DeepSeek V4 Pro's recipe would cost >$100M
- Evaluated bits-per-byte loss on heldout code, papers and math vs open base models from DeepSeek, Moonshot and NVIDIA
- Team believes pretraining + agentic RL + long context suffice for superhuman coding agents and automating AI R&D
orange notes Magic reported a similar plot (505x gains) two months ago, quipping we may not need so many GPUs.
More from Infra
- Why No One Can Replicate Jensen Huang's Asian GPU Supply Chain — _onionesque · 2026-11-15
- Running 3 AI agents in parallel: even 32GB machines hit memory limits — PawelHuryn · 2026-11-15
- Curated 10-resource path to learn LLM distributed training from scratch — East-Muffin-6472 · 2026-11-15
- Report: Nvidia's interim CEO reviews AI customer financing, $300B OpenAI Ohio chip deal in doubt — SumitGup · 2026-11-15
- Skip the $3,000 Mac mini: a $1,000 renewed Dell micro PC works fine for agents — PawelHuryn · 2026-11-15
- Pirate Face mirrors open models as torrents, already indexing 28,890 magnet links — zainhas · 2026-11-15