Defending METR: public data can't second-guess its AI speedup findings
peterwildeford · x · 2026-11-11
Peter Wildeford pushes back on people using public data to second-guess METR's AI R&D speedup conclusions, noting METR drew on far more non-public evidence — internal model benchmarks, intermediate measures, algorithmic progress estimates, bottleneck details and staff interviews — much of it shared with METR but redacted from the public report. He adds Anthropic should redact less.
More from Safety
- Researchers compromise latest Codex: prompt injection to sandbox escape to npm supply chain — rez0__ · 2026-11-11
- Record a walk, let Claude Code run the experiment: a 3-step research workflow — DimitrisPapail · 2026-11-11
- US data vendors accused of selling RL environments to Chinese military-linked firms — xeophon · 2026-11-11
- LMArena co-founder explains AI evaluation methodology and policy in new podcast — MattPerault · 2026-11-11
- Australia's review of an OpenAI agent's unauthorized Medicare access exposes the agent accountability problem — cmb24k · 2026-11-11
- Adversarial RL envs for backdoor insertion vs. detection: where does it converge? — DimitrisPapail · 2026-11-11