METR cross-domain chart: time horizon levels differ by domain but growth rates are similar
BasilHalperin · x · 2026-11-09
Offering explanation (a) for the slow-looking OpenAI internal numbers, Halperin points to Kwa's METR cross-domain analysis: different domains (SWE-bench, LiveCodeBench, GPQA, Tesla FSD, OSWorld, etc.) have different time-horizon intercepts but similar slopes — software tasks may just be a hard domain rather than evidence of a general slowdown.
More from AGI Musings
- Miyazaki: Young anime artists who only study anime are 'spinning wheels' — threepointone · 2026-11-09
- Dev Headcount Down, Customer-Facing Roles 10x: The Post-LLM Job Map — kylegawley · 2026-11-09
- ryolu_ releases full Shanghai talk: when AI can do anything, do we know what we want? — ryolu_ · 2026-11-09
- Australia Sees 108,300 Redundancies in Two Months, Highest Since COVID Amid AI Push — gaganghotra_ · 2026-11-09
- Productivity gains in OpenAI's data are actually declining over time, researcher finds — austinc3301 · 2026-11-09
- Ramez: OpenAI's Productivity Gain Per ECI Point Appears to Decline Over Time — ramez · 2026-11-09