Tang Jie on LLMs: Post-Training is More Crucial
dotey · x · 2026-07-18
This repost summarizes Tang Jie's extensive thoughts on LLM development in 2025. The core message: pre-training remains important but is no longer the only protagonist; what truly determines a model's real-world applicability is mid-to-late stage training that activates capabilities for long-tail scenarios and practical tasks.\n\nHe also makes a strong assertion: the first principle of AI applications shouldn't be "building new apps," but rather "replacing human labor." Following this logic, when building applications, one should prioritize identifying which roles and workflows are best suited for AI takeover, rather than just chasing superficial product innovation.\n\nThe post also emphasizes that many current models have become "over-specialized" just to ace benchmarks, making them unstable in complex real-world scenarios. Consequently, post-training, alignment, and training oriented towards real-world tasks will become increasingly critical.
More from AGI Musings
- Benjamin Todd: AI progress beat superforecasters over 18 months — ben_j_todd · 2026-10-26
- AI researchers guessed robots wouldn't outrun humans until 2034 — it already happened — ben_j_todd · 2026-10-26
- AI folks, stop saying "biology" when you mean drug development — KevinKaichuang · 2026-10-26
- ESR on the Bugpocalypse: locking down frontier models only hands the game to Chinese open weights — colinmcnamara · 2026-10-26
- Anthropic: junior tech hires fell from 27% to 9% as agent hours hit 6x employee hours — thione · 2026-10-26
- Anthropic's Jascha: you may have only 1–5 years of intellectual contribution before AGI — GabGarrett · 2026-10-26