Debate over METR's 2x speedup claim: does it lean on private information?
JacquesThibs · x · 2026-11-08
JacquesThibs debates JeffLadish over METR's report claiming AI doubles task-completion speed. He argues the 2x number may hold within error bars, but only under different AECI benchmark fit assumptions — which would mean METR leans far more on private, unpublished information than the report admits. He adds that "METR has information we don't" only supports the overall 2x figure, not the 1.9x slope increase in the AECI fit.
Related event: Researchers Debate Reliability of METR 2x Speedup Claim(2 posts)→
More from Research
- Latent Sparse Attention paper: indexing eats 98% of prefill at 10M tokens, LSA speeds one layer 22.7x — burny_tech · 2026-11-08
- Double-exponential fit suggests model time horizons could hit ~80 work-years by end of 2027 — scaling01 · 2026-11-08
- ACM policy lets authors share anonymous reviews, no barrier to crowdsourcing — IanArawjo · 2026-11-08
- CHI researcher proposes open peer-review database, wants LLM reviewers banned — IanArawjo · 2026-11-08
- AI capability growth as a tug of war: recursive self-improvement vs diminishing returns — CFGeek · 2026-11-08
- Agent S4 lets agents evolve reusable computer-use interfaces from their own task experience — burkov · 2026-11-08