Stanford prof Anshul Kundaje publicly calls Hugging Face's open-model benchmarks 'extremely problematic'
anshulkundaje · x · 2026-10-23
Stanford professor Anshul Kundaje publicly asked Hugging Face CEO Clement Delangue for a conversation about the company's evals, saying the benchmarks HF has been using for its open-model evaluations are "extremely problematic" — and offering to point to much better alternatives. He notes this isn't his first critique of this line of work. Given his standing in ML for genomics, the debate over open-model eval methodology is worth watching.
Related event: Stanford Professor Critiques Hugging Face Benchmarks; CEO Concedes Flaws(2 posts)→
More from Models
- ChatGPT web can actually listen to audio, but the same model can't via API or Codex — Possible_Statement84 · 2026-10-23
- MiniMax M3.1 Flash free for 3 days on 0G, with 1M context and 512K output — MiniMax_AI · 2026-10-23
- TLA+ expert corrects Grok: TLC only exhausts finite state spaces, not all behaviors — tianyin_xu · 2026-10-23
- OpenAI blocks manual reset with 13% usage left: 'no reset needed right now' — AccomplishedSugar490 · 2026-10-23
- Meta's 816B Muse Spark Open scores 39, losing to 180B Qwen3.8-Flash-Next at 40 — BTA_Labs · 2026-10-23
- Gemini refuses to generate image in Docs, blames document size limit — viksit · 2026-10-23