On the METR graph: insiders seeing AECI tasks is weak evidence, not zero evidence
gleech · x · 2026-11-06
gleech argues in a debate that METR researchers seeing the AECI benchmark components and sample tasks and being 'freaked out' isn't scientific evidence, but it also shouldn't be weighted as zero evidence.
Related event: METR's New Capability Chart Sparks Debate Over Leaked AECI Benchmarks(2 posts)→
More from Models
- METR data: Anthropic now pretrains every 45-60 days as capability gains double — zephyr_z9 · 2026-11-06
- Custom GPTs die Dec 11: if you use someone else's GPT, you can't migrate it — Ok_Negotiation_2587 · 2026-11-06
- Kimi K3.1 and Tencent Hy4 accused of gutting cybersecurity skills from open-weight models — Fresh_Sun_1017 · 2026-11-06
- Daily AI brief: GPT-6.2 imminent, Claude Personal Finance beta, METR says Anthropic progress up 1.9x — testingcatalog · 2026-11-06
- METR report researcher quotes lean bullish: Opus 4.5 coding plus faster experiment cycles — gleech · 2026-11-06
- Epoch AI review: OSWorld tasks are simple, unstable, and barely test GUI use — gleech · 2026-11-06