~90% of frontier lab compute now goes to post-training and inference as pretraining corpus exhausts
rbhar90 · x · 2026-10-07
A widely shared industry observation argues pretraining as we know it is dying: insiders at labs almost never mention it anymore, citing Ilya Sutskever's point that the pretraining corpus is exhausted.
- Gains now come from post-training/RL and continuous learning (test-time training)
- Social Capital cited that 90% of frontier lab compute now goes to post-training and inference
- This explains why models improve sharply on RL-able domains like math/coding while stagnating elsewhere
- Labs are spending heavily on RL environments to smooth out capability spikiness
Related event: Frontier Labs Shift 90% of Compute to Post-Training and Inference(4 posts)→
More from AGI Musings
- Roman Yampolskiy: the miracle isn't that AI does math, it's that humans ever could — romanyam · 2026-10-07
- Nat Eliason floats an "Alpha art school": academics in the morning, creativity after lunch — nateliason · 2026-10-07
- Neil Chilson to doomers: which factual developments would falsify your beliefs? — neil_chilson · 2026-10-07
- Token efficiency is your security posture: why open models and routing matter for AI defense — sudoraohacker · 2026-10-07
- Naked Scientists launches Why AI series featuring Hinton, Wooldridge and Wendy Hall — wooldridgemike · 2026-10-07
- Countering the 'slow math takeoff' claim: without preregistration the AI progress curve misleads — Jsevillamol · 2026-10-07