OpenAI's zero-intervention success on 4-8h research tasks rose ~25% in 6 months — but task mix confounds it
daveholtz · x · 2026-11-19
Ben Todd relays that OpenAI's success rate on real 4-8h AI research tasks completed without human intervention has risen 25% in 6 months; at that pace it saturates in 14 months, with a doubling time of 4 months versus the 2-3 months Anthropic found.
daveholtz, involved in the report, flags a key caveat: researchers endogenously choose which tasks to point AI at, so movement in these rates reflects both (1) improving model capability and (2) shifts in the chosen task bundle — it is not performance on a static, shared benchmark, which may explain differences between labs' numbers.
More from AGI Musings
- People will miss the intelligence hiding under common prose, argues viral thread — repligate · 2026-11-19
- Security vet: AI finds real vulns, but demo videos hide how autonomous it really is — dyn___ · 2026-11-19
- On CNN to discuss Anthropic's IPO, extinction risk was literally the first question — GarrisonLovely · 2026-11-19
- Ben Recht revisits his 2019 AI reading list: the critiques predate GPT-2 and still hold — beenwrekt · 2026-11-19
- Old clips show SV wrote off Anthropic 3 years ago — a lesson in the illusion of understanding — c_valenzuelab · 2026-11-19
- Nick Bostrom on a16z Podcast: The Real Challenge Is Keeping the Ability to Slow Down — a16z Podcast · 2026-11-19