Leaderboards keep rotating benchmarks because of goodharting, argues xeophon
xeophon · x · 2026-11-09
Replying to llmwizard, xeophon notes that collections like the Artificial Analysis Intelligence Index — the benchmarks people obsess over — rotate their benchmark selections constantly, largely due to goodharting. "You know and I know, but others don't."
More from Models
- François Fleuret teases mystery model: 'v1.3 is dead, v1.4 was silly, glory to v1.5!' — francoisfleuret · 2026-11-09
- ValsAI reports critical safety flaw in frontier lab models, may delay releases — koltregaskes · 2026-11-09
- Tesla Optimus engineer builds GROXIE, a countertop robot assistant powered by Grok — Scobleizer · 2026-11-09
- Mistral Large 4 lands: 1.05T-param MoE, best Western open weights but only mid-pack — maier_ak · 2026-11-09
- ChatGPT-User crawls drop 81% as OpenAI shifts to its own index — gaganghotra_ · 2026-11-09
- LLMs show "insane regression" in intelligence when discussing cell therapy, says blogger — owl_posting · 2026-11-09