Sarah Hooker: GLM 5.3's leap shows pretraining is dead, post-training is the frontier
khademinori · x · 2026-10-19
Anthropic researcher Sarah Hooker argues that GLM 5.3's gains coming purely from post-training is a symptom of pretraining's decline: as transformers saturate, pretraining yields mostly data-recipe tweaks rather than stepwise leaps. Innovation is shifting to post-training and test-time compute — which is good for competition, since pretraining concentrated resources in a few labs, while post-training is far more GPU-efficient and lets companies with proprietary data match closed models. The cited case: GLM 5.3 was entirely post-trained from its predecessor, with night-and-day cyber capability gains over 5.2.
More from Models
- Why Is ChatGPT Instant but Local Qwen Slower? Reddit Debates Inference Stacks — dhtp2018 · 2026-10-19
- Andrew Carr: it took 3 tries to convince Argon it could output 1M tokens — andrew_n_carr · 2026-10-19
- Mathematician claims LLMs only interpolate, can't discover new math like Riemann — burny_tech · 2026-10-19
- Timezone conversion was my last use case for Google — and LLMs now handle it — shauseth · 2026-10-19
- Qwen Image 2.1's built-in prompt enhancer is unusable: 12k tokens still not enough — dtdisapointingresult · 2026-10-19
- 93-Model Benchmark Over 1.5 Years: Distillation + Quantization + RL Hurts General AI — Afinetheorem · 2026-10-19