ARC-AGI-3 perfect score challenged: public set was designed to be easier
JFPuget · x · 2026-09-01
François Chollet pushed back on a vendor's claim that its general-purpose coding agent scored 100% on ARC-AGI-3. Quoting the ARC 3 paper from March, he noted the public set was intentionally made easier for both humans and AI, merely to demonstrate the environment format — something the announcement tweet omitted.
JFPuget replied that they make no AGI progress claims with this result, and countered that the paper itself doesn't state the score applies only to public evaluation games.
Related event: Coding Agent's ARC-AGI-3 100% Claim Rebutted by Chollet(6 posts)→
More from Models
- Swapping RLM sub-calls for Jev: half the cost, 6x faster at 1M tokens — lateinteraction · 2026-10-16
- Stanford users say OpenAI cut ChatGPT Edu Pro usage 50% mid-contract — DJiafei · 2026-10-16
- MiniCPM-V 4.5 MoE reportedly hits 13+ tokens/s on iPhone with just 3GB memory — heyshrutimishra · 2026-10-16
- IQuest-Q1 writes a 629-line submarine game from one prompt in 2 minutes — Div_pradeep · 2026-10-16
- Trying Grok bot surfaces an unpaid Cursor balance, blocking account merging — haydendevs · 2026-10-16
- TensorFold 1.0.12 runs 120B vision model on 1-2 DGX Sparks, adds Mac image input — HankYeomans · 2026-10-16