DeepMind researcher: next-gen model gains come from bigger pretrain + data/algorithm gains, not compute speedup
tobyordoxford · x · 2026-11-07
Toby Ord (DeepMind researcher) interprets recent next-gen model rumors on X: the capability jump isn't training compute acceleration, but a mix of a long, steady sequence of larger pre-trains plus many data-quality and pre-training algorithmic improvements since the last big run.
His best guess: a large new pre-train worth 5 AECI, followed by a 1.4x speed improvement — and he doesn't think the original claims rule that out.
More from Models
- LMArena launches Code Arena with WebDev categories for head-to-head coding tests — arena · 2026-11-07
- Empero's Manta claims a 100-million-token context window in a single prompt — emdeka87 · 2026-11-07
- Team bans commas and noun-phrase clustering to scrub Claude's AI writing tells — layer07_yuxi · 2026-11-07
- 68-hour pretraining: Chinese open-weight models run circles around Western open SoTA — teortaxesTex · 2026-11-07
- VIALS benchmark: frontier models flunk 161 life-sciences image interpretation tasks — niloofar_mire · 2026-11-07
- MIT's Drift Calibrator visualizes sycophancy and empathy drift live during AI chats — patpat_mit · 2026-11-07