Tang Jie on Scaling Law: Optimal split between data and compute
AccBalanced · x · 2026-08-20
Tsinghua's Tang Jie discussed Scaling Law, arguing that focusing solely on parameter count is one-sided. Parameters must be evaluated alongside data volume, compute allocation, and deployment conditions. The article reviews the evolution from Kaplan (2020), suggesting parameters grow faster than data (2.7:1), to Hoffmann (2022), which found the compute-optimal split is closer to "20 tokens per parameter" after experimenting with 400 models.
Republisher Perry Metzger contrasted this deep technical engagement and openness with American AI executives (like Dario) who publish "neurotic" manifestos while keeping their companies secretive, suggesting this is a significant potential advantage for China.
More from Companies & People
- Researcher criticizes companies publishing obviously AI-written blog posts as PR farming — nrehiew_ · 2026-10-16
- UZH Robotics & Perception Group opens funded PhD/postdoc spots in world models and spatial AI — HildeKuehne · 2026-10-16
- Anthropic commits $5M to help mathematicians navigate AI; Simons Foundation pledges $15M total — AnthropicAI · 2026-10-16
- Chart: Meta went 492 days without an open-weight release before Muse Spark Open — demian_ai · 2026-10-16
- Accenture bills billions to deploy AI for clients yet can't make its own staff productive — matt_slotnick · 2026-10-16
- Meta's head of AI slams AI benchmarks: 'outsourcing product thinking to viral leaderboards' — benno_krojer · 2026-10-16