Zhipu's Tang Jie: Parameter Count No Longer the Sole Scaling Metric
量子位 · wechat · 2026-08-19
Zhipu AI founder Tang Jie reviewed the evolution of LLM Scaling Laws, arguing that parameter count alone is insufficient to gauge model level; data volume, compute allocation, and deployment conditions must also be considered.
Three Shifts in Scaling Strategy
- Parameter Priority: Early Kaplan research suggested parameters should grow faster than data, sparking the trillion-parameter race.
- Data Compensation: DeepMind's Chinchilla laws revealed models were over-parameterized and under-trained, establishing a 20 Token/parameter optimal ratio.
- Lifecycle Cost: Factoring in high-frequency inference costs shifted the optimum again. Models like Llama 2 and Gemma 2 are "over-trained" far beyond Chinchilla standards.
Scaling in the MoE Era
Tang distinguishes between "total parameters" (knowledge repository) and "active parameters" (reasoning capacity). Using GLM-5.3 as an example, keeping the architecture unchanged but focusing on long-chain task post-training led to significant gains on benchmarks like Terminal-Bench. The conclusion is that Scaling has multiple knobs (params, pre-training data, post-training) that don't need turning simultaneously. The industry focus is shifting from knowledge base size to the ability to complete complex tasks.
More from Companies & People
- Researcher criticizes companies publishing obviously AI-written blog posts as PR farming — nrehiew_ · 2026-10-16
- UZH Robotics & Perception Group opens funded PhD/postdoc spots in world models and spatial AI — HildeKuehne · 2026-10-16
- Anthropic commits $5M to help mathematicians navigate AI; Simons Foundation pledges $15M total — AnthropicAI · 2026-10-16
- Chart: Meta went 492 days without an open-weight release before Muse Spark Open — demian_ai · 2026-10-16
- Accenture bills billions to deploy AI for clients yet can't make its own staff productive — matt_slotnick · 2026-10-16
- Meta's head of AI slams AI benchmarks: 'outsourcing product thinking to viral leaderboards' — benno_krojer · 2026-10-16