GDPval benchmark under fire: first seed artifact in public split looks off
vincent_koc · x · 2026-10-22
A developer examined the public split of the GDPval benchmark for the first time and found the very first seed artifact they checked was problematic.
GDPval claims its tasks are "constructed from the representative work of industry professionals with an average of 14 years of experience," positioning it as a frontier benchmark — and its public split is a key benchmark on Artificial Analysis. The post questions the data quality behind such headline evals, with the comment "never look under the hood of an eval."
More from Models
- Carbon checkpoint on LMArena updated, Gemini web and app change in sync — lyraxana · 2026-10-22
- KRAFTON's Raon-1-mini-preview Trends on Hugging Face — KRAFTON · 2026-10-22
- Gemini 4 Carbon Checkpoint Wows at 3D Generation: Opus 5.5 Level? — WorldofAI · 2026-10-22
- Hand-picked ImageNet photos break InceptionV1 neuron 4e:55's 'pointy vs round' rule — livgorton · 2026-10-22
- Rumor: Anthropic's next model could land today or tomorrow per release cadence — himanshustwts · 2026-10-22
- Lighthouse Benchmark: Gemini Argon Runs 8x Cheaper Than Claude Opus — cgarciae88 · 2026-10-22