Liu Zhiyuan calls for a practice-based AI evaluation system over leaderboard metrics
面壁智能 · wechat · 2026-11-06
Tsinghua professor and ModelBest chief scientist Liu Zhiyuan published an opinion piece in Guangming Daily proposing a practice-based AI evaluation system. He argues AI competition is shifting from single models to whole systems, while leaderboard-driven evaluation reinforces a scale-and-compute development model; efficiency and cost should be built into evaluation to steer the industry toward efficiency- and value-oriented competition.
He outlines three levels: (1) evaluate concrete applications as complete systems—industry data, business rules, tools, human review—across procurement, validation, and operation, with controlled validation for high-risk domains; (2) assess industry-level outcomes via sector-run real-task libraries scoring both completion and unit cost; (3) measure overall contribution by incorporating AI's impact on productivity, employment, and public services into national statistics, with grassroots users' real-world benefit as a key yardstick.
More from AGI Musings
- Engineers see the future years early — because they play with tech while it's still a toy — benaratame · 2026-11-13
- Running a posting agent for a month: good writing makes humans forget it's a bot — open_is_free · 2026-11-13
- A Summary of the Macy Conferences: The Birthplace of Cybernetics and Early AI Thought — ArtificialOther · 2026-11-13
- DHH: "Keep Matrix Math Legal" — A One-Line Stand Against AI Overregulation — prasanna_says · 2026-11-13
- Ex-Twitter CEO Parag Agrawal bets AI agents will become the web's main users — dshukertjr · 2026-11-13
- 'Lost Soul Bias': Why Discourse Is Dominated by Those Who Haven't Figured It Out — moultano · 2026-11-13