Shunyu Yao at Tsinghua: how Tencent builds Hunyuan, anti-benchmark-gaming, and embodied AI's GPT-3.5 moment
量子位 · wechat · 2026-11-20
Tencent Hunyuan lead Shunyu Yao gave a 2+ hour talk at his alma mater Tsinghua, covering how his team trains models and his view of the field.
Training methodology
- Pretraining uses a "Scaling Ladder": run experiments at several small scales first, verify gains transfer, then scale up.
- Repeats of scarce data risk memorization; synthetic data must hold quality. Post-training is back to plain Policy Gradient (incl. GRPO) for fewer assumptions and better scaling; evals should be designed before training data and reflect real user tasks.
Anti-benchmark-gaming & culture
- "Overfitting a leaderboard hurts yourself first." Hunyuan hires for LowEgo, honesty, and a Taste-and-Trust culture. If your model matches GPT/Claude but earns 1% of their revenue, something is wrong downstream.
Industry calls
- Embodied AI may hit its "GPT-3.5 moment" in perhaps six months to a year; scalability, not niche models, decides the winning route. Small models are far from their ceiling; the data wall isn't near. Long-term, compute may not decide the US–China gap — data and physical environments matter more.
- Founder test: "If the next GPT ships tomorrow, do you sleep better or worse?"
Open-source safety
- Distinguishes Safety (model's own dangerous behavior) from Security (misuse by people); the urgent one is Security. Responsible open source requires layered defenses across training, evals, and deployment, with security tools prioritized for defenders.
Also covers using AI for alignment and limits of Transformers for recursive self-improvement.
More from AGI Musings
- The hardest work skill by 2028 won't be prompting — it's writing goals you can accept — evisoft · 2026-11-20
- Does AI make building small profitable businesses easy but big durable ones harder? — vaibhavbetter · 2026-11-20
- US and China Account for 74% of AI Models, Raising Tech Dependency Concerns — mikeflache · 2026-11-20
- Google Documents First Fully Autonomous Attack System, Exploit Agent Turns Patches Into Attack Maps — mikeflache · 2026-11-20
- Nokia Bell Labs talk: AI and the Restructuring of Jobs, Nov 23 — danielequercia · 2026-11-20
- Anthropic's 5,300 founders and employees now worth more than bottom third of US households — jathansadowski · 2026-11-20