Anthropic S-1 claims Claude jumped 40% to 90% on open-ended engineering tasks in 9 months
trevposts · x · 2026-11-09
Caleb Watney highlights an overlooked Anthropic S-1 claim: Claude's success rate on open-ended, judgment-heavy engineering tasks went from 40% to 90% in nine months — arguably a bigger update than METR's 2x speedup, if the benchmark holds up. The linked essay "The Goodhart Singularity" by Tom Reed argues automating AI R&D alone is insufficient for superintelligence: whether progress in verifiable domains transfers to fuzzy, open-ended tasks is the key crux.
More from AGI Musings
- Joshua Saxe: offline evals aren't progress — measure AI by labor productivity and real-world harm — typewriters · 2026-11-09
- Hassabis: AI should take over teaching, classrooms keep only what machines can't do — xiaosun86 · 2026-11-09
- You can't compete with the labs on RSI without nation-state compute, researcher argues — generativist · 2026-11-09
- Fear Humans Who Stop Thinking Long Before Machines Ever Could — yunta_tsai · 2026-11-09
- Researcher: Even as AI reshapes everything, serious output verification still matters — srchvrs · 2026-11-09
- Safety researcher: third-party ecosystem must scale AGI assessments before superhuman AI arrives — austinc3301 · 2026-11-09