Meta's head of AI slams AI benchmarks: 'outsourcing product thinking to viral leaderboards'
benno_krojer · x · 2026-10-16
Meta's head of AI slammed AI benchmarks: "it's some PhD student who made these benchmarks," "you're kind of outsourcing all of your product thinking to this viral benchmark," "so clearly that's wrong."
Quoter aleshuber argues the industry moved so fast that proper benchmarking never had time to mature: companies used whatever existed, competitors got locked onto the same benchmarks, and some went on to become de facto standards—meaning today's measurement of AI progress carries a lot of randomness. Kenneth Marino, who both made a viral benchmark as a PhD student and worked on product evals for Gemini at DeepMind, responded with a thread saying he has a unique perspective on this.
Related event: Meta AI Chief Slams Industry Benchmarks(2 posts)→
More from Companies & People
- Dario Amodei says AI better than humans at almost everything may be 2-3 years away — PolarBearby · 2026-10-16
- Jensen Huang: Musk's AI buildout moved from K's to M's, now a CSP-tier NVIDIA customer — elonmusk · 2026-10-16
- Vercel CEO Rauch and Shopify CEO Tobi Lütke share stage at Ship SF — evilrabbit_ · 2026-10-16
- Australia's AERO guidance: avoid AI entirely in first three years of primary school — lmoroney · 2026-10-16
- Alexandr Wang: Meta Superintelligence Labs is in South Bay, and strangers talking to you means something's up — HanchungLee · 2026-10-16
- Accenture caps Claude reimbursement at $29/month while employees spend up to $1,000 — inductionheads · 2026-10-16