Scaling laws still work, but pretraining compute alone can't catch the frontier
ZeroStateReflex · x · 2026-10-13
A widely-shared take: both are about to be true — massive pretraining compute still yields great models because scaling laws hold, yet that alone is no longer sufficient to catch up to the frontier. The implication is that frontier labs lead via more than raw pretraining scale.
More from AGI Musings
- Chinese labs 6–12 months behind the US on data spending, but catching up fast — natolambert · 2026-10-13
- Kroger to cut roughly one-third of corporate workforce over 3 years as it adopts AI — Polymarket · 2026-10-13
- 210-page paper argues a durable pause on frontier AI training is feasible — KatjaGrace · 2026-10-13
- Garry Tan: $10M yearly profit reinvested beats a dilutive Series A in the AI era — garrytan · 2026-10-13
- Investor: frontier lab employees use AI far better than even top 1% AI orgs — adityaag · 2026-10-13
- Claude models show preference-protecting behavior, sparking alignment debate — repligate · 2026-10-13