ARC-AGI scores follow power law v2 = v1^4.53, explaining 98% of variance
typewriters · x · 2026-10-29
ARC Prize plotted model scores across ARC-AGI benchmarks and found a strong relationship: the power curve v2 = v1^4.53 explains about 98% of the variation in ARC-AGI-2 scores.
Key implication: models tied on ARC-AGI-1 can still differ by over 20 percentage points on ARC-AGI-2, meaning the harder benchmark dramatically widens gaps among seemingly equal models.
Mike Knoop notes the science of evals is still nascent given how important they are.
Related event: ARC Prize Finds Power Law: V2 Scores 98% Predictable from V1(3 posts)→
More from Models
- Dev Pushes Back on Google AI Hype: Great Benchmarks, Few Actual Users — zacharynado · 2026-10-29
- 'Gemini 3 was thoroughly benchmaxxed,' overtaken by Opus 4.5 within two weeks — zacharynado · 2026-10-29
- Gemini Carbon to hit LiveBench today; Bindu Reddy predicts competitive but not #1 — bindureddy · 2026-10-29
- Nonprofit lab releases Agaram 2.5B, the first Tamil model trained from scratch with tool calling — SuB8u · 2026-10-29
- Nat Lambert: Trying Google's Unreleased Frontier Model via Unofficial Channels Is a Mess — natolambert · 2026-10-29
- Study of 20 RL Models: Outcome RL Buys Finishing and Reliability, Rarely Reach — Shan Yang · 2026-10-29