Vik Paruchuri: Even Experts Struggle to Tell Good Benchmarks from Bad Ones
VikParuchuri · x · 2026-11-17
Marker/元数据工具作者、Dyanimolous 创始人 Vik Paruchuri 回应 Hamel Husain 谈基准测试问题,给出四点看法:亲自看数据确实是关键;大多数基准设计得还不错;但即便是专家也很难区分设计好与设计差的基准;行业内存在大量粉饰结果与基准的动机。
More from AGI Musings
- Ben Bajarin: enterprise AI adoption is the only conviction that matters for the decade — BenBajarin · 2026-11-17
- LangChain founder: agents are shifting from executing tasks to owning goals — hwchase17 · 2026-11-17
- Paul Christiano: 1-in-3 chance of government-toppling superintelligence within a year — Dwarkesh Patel · 2026-11-17
- Sam Altman on stage: "We didn't have AGI. Now we do" — and OpenAI is withholding self-improving AI — imjustnewatai · 2026-11-17
- Report: US Has 9x China's Compute, So It Can Afford to Pace the AI Frontier — trevposts · 2026-11-17
- Open-source models now serve 30% of tokens, growing as fast as closed frontier AI — ivan_bezdomny · 2026-11-17