OpenAI's p50/p80 time horizon ratio is ~4x larger than METR's
BasilHalperin · x · 2026-11-09
Halperin highlights a striking discrepancy: in METR data the 50% time horizon is 4x the 80% horizon, but in OpenAI's internal data the 80% horizon is 15x shorter than the 50% one. He's unsure what to make of it — possibly related to low- vs high-context human baselines, or p50/p80 ratios varying substantially by field.
More from AGI Musings
- Miyazaki: Young anime artists who only study anime are 'spinning wheels' — threepointone · 2026-11-09
- Dev Headcount Down, Customer-Facing Roles 10x: The Post-LLM Job Map — kylegawley · 2026-11-09
- ryolu_ releases full Shanghai talk: when AI can do anything, do we know what we want? — ryolu_ · 2026-11-09
- Australia Sees 108,300 Redundancies in Two Months, Highest Since COVID Amid AI Push — gaganghotra_ · 2026-11-09
- Productivity gains in OpenAI's data are actually declining over time, researcher finds — austinc3301 · 2026-11-09
- Ramez: OpenAI's Productivity Gain Per ECI Point Appears to Decline Over Time — ramez · 2026-11-09