ARC-AGI-3's non-standard scoring under fire as GPT-6 hits ~100%
peterwildeford · x · 2026-09-14
- Peter Wildeford (ex-ARC researcher) remarked that ARC-AGI-3's scoring system is "way sillier" than he realized.
- Sterling Croxton notes ARC-AGI-3 uses a totally non-standard way of calculating scores, making a hypothetical jump to 100% by models like GPT-6 the most likely outcome regardless of true capability gains.
- Critics argue ARC should never have adopted this methodology, and it has now "blown up in their faces."
- Takeaway: ARC-AGI-3 scores may not be comparable to previous benchmarks and should be interpreted cautiously.
More from Models
- Leak: Google Reportedly Has Another 'Hella Chonky' Model After Carbon — iruletheworldmo · 2026-10-30
- Users report across-the-board GPT quality drop: weaker writing and broken instruction following — Kyrannio · 2026-10-30
- Leak claims Gemini Argon hits Pro trial from November 2nd (unverified) — Emotional-Company847 · 2026-10-30
- OpenAI retiring waves of legacy models: GPT-5 family dies Dec 11, GPT-4.1 by Jan 2027 — ThePeterMick · 2026-10-30
- DeepMind CEO Demis Hassabis tells Musk: testing with trusted partners, coming soon — demishassabis · 2026-10-30
- Adobe stands firm on not training on customer data, using licensed stock instead — omooretweets · 2026-10-30