User Accuses Anthropic of Gaming ARC-AGI-3 by Training Specifically on Benchmark Patterns
VraserX · x · 2026-07-25
A user on X accused Anthropic of gaming the ARC-AGI-3 benchmark. The post claims that Anthropic essentially trained specifically on the benchmark’s puzzle patterns, turning visual reasoning tasks into explicit algebra and drilling the exact strategy. The user argues that this is not general intelligence but merely benchmark optimization, concluding that benchmark scores barely mean anything anymore.
Related event: Opus 5's High ARC-AGI-3 Score Sparks Cheating and Overfitting Controversy(11 posts)→
More from Models
- MissionReport benchmark: only 3 of 540 AI-generated government reports pass all criteria — Exp_Mark · 2026-10-27
- Local beats cloud: Qwen 3.8 FN handles private student data as well as frontier models — dr_lm · 2026-10-27
- Aidan Gomez: model's CoT said no tool calls right before making them — aidangomez · 2026-10-27
- Nat Lambert: Cohere's model is narrow RL training on internal infra ill-suited for open weights — natolambert · 2026-10-27
- OpenAI teases DOTS is coming to Europe 'soon' — mark_k · 2026-10-27
- Clifford Pickover challenges AI to decipher a message he encoded in Norse runes — pickover · 2026-10-27