ARC team: v1 hits ceiling effect as v2 extends headroom for frontier models
MindsAI_Jack · x · 2026-10-29
In an ARC discussion, MindsAIJack notes ARC v1 is hitting a ceiling effect, and frontier models may be nearing it on v2/v3. Greg Kamradt responds that plotting v1 vs v2 scores shows smooth progression, v2 extended the original ARC-AGI format's usefulness at higher model levels, and that with existing data they could infer a difficulty per task — potentially enabling a v2.5 to push the format further (no plans currently).
Related event: ARC Prize Finds Power Law: V2 Scores 98% Predictable from V1(3 posts)→
More from Models
- theo slams new model as worse than Composer 2 at 4x the cost, calling it Llama 4 tier — zacharynado · 2026-10-29
- Community speculates Gemini's new tier naming: Carbon, Argon, Flash — koltregaskes · 2026-10-29
- Kaggle kicks off Bench Summit, arguing today's AI benchmarks fail on relevance, compute and verification — MeganRisdal · 2026-10-29
- SWE-sweep appears in a Google DeepMind frontier model card within a month — OfirPress · 2026-10-29
- Dev Pushes Back on Google AI Hype: Great Benchmarks, Few Actual Users — zacharynado · 2026-10-29
- 'Gemini 3 was thoroughly benchmaxxed,' overtaken by Opus 4.5 within two weeks — zacharynado · 2026-10-29