Less-verifiable benchmarks speed up just as fast: 1.64x vs 1.67x in Anthropic data
ben_j_todd · x · 2026-11-07
Replying to his own question about whether AI acceleration only applies to verifiable tasks, benjtodd shares a chart showing Anthropic models improved 1.64x faster on less-verifiable ECI benchmarks versus 1.67x on more-verifiable ones — wherever data exists, less-verifiable tasks are improving just as quickly. Data from Karthik Tadepalli's "Anthropic's AI acceleration is probably real".
More from AGI Musings
- China's CAS five-year plan makes AI for Science a top priority: robots vs compute race — teortaxesTex · 2026-11-07
- China's fully electric driverless tractor is already farming fields in Hebei, nationwide rollout planned — lukas_m_ziegler · 2026-11-07
- Researchers joke March 2027 is the deadline before AI catches up with AI research itself — DrDatta_AIIMS · 2026-11-07
- The Agent Cloud is actually two clouds: one where models think, one where agents work — AccBalanced · 2026-11-07
- Author who wrote for years questions what 'writer' means amid flood of AI books — Fun_Improvement9440 · 2026-11-07
- Ethan Mollick: LLMs can 10x investigative journalism research — emollick · 2026-11-07