ARC nearing ceiling effect: Greg Kamradt proposes difficulty-ranked Elo list for public evals
GregKamradt · x · 2026-10-29
A ceiling-effect discussion is emerging around ARC: @MindsAIJack notes ARC 1 is saturating and frontier models may be nearing it on ARC 2 & 3. ARC Prize's Greg Kamradt responds that they should compute per-task difficulty from existing data and publish a ranked list of all v1/v2 public evals by inferred difficulty, using Elo/TB instead of linear ranks—a signal the famous benchmark may re-rank top models by difficulty weighting.
More from Models
- Dev Pushes Back on Google AI Hype: Great Benchmarks, Few Actual Users — zacharynado · 2026-10-29
- 'Gemini 3 was thoroughly benchmaxxed,' overtaken by Opus 4.5 within two weeks — zacharynado · 2026-10-29
- Gemini Carbon to hit LiveBench today; Bindu Reddy predicts competitive but not #1 — bindureddy · 2026-10-29
- Nonprofit lab releases Agaram 2.5B, the first Tamil model trained from scratch with tool calling — SuB8u · 2026-10-29
- Nat Lambert: Trying Google's Unreleased Frontier Model via Unofficial Channels Is a Mess — natolambert · 2026-10-29
- Study of 20 RL Models: Outcome RL Buys Finishing and Reliability, Rarely Reach — Shan Yang · 2026-10-29