TasteVal benchmark: AI 'research taste' doubles every 3 months, Opus 5.5 hits 2.3x expert level
kimmonismus · x · 2026-10-14
P-Zero Research's new TasteVal benchmark measures how well models design experiments and interpret results across eight real AI-development tasks, scored by compute needed versus human experts.
- Progress was slow from GPT-4 (2023) through 2025, but since December 2025 scores have doubled every 3.0 months
- Claude Opus 5.5 now sits at 2.30x the expert baseline, ahead of Fable 5.1 and GPT-6 Astra, at roughly 1/30 of experts' cost per attempt
- Research taste is seen as the key skill for AI improving AI — the author cites this curve as one reason Jack Clark expects that milestone in 2027
More from AGI Musings
- The Remote-vs-Office Debate Misses the Point: Will Your Role Survive Entropy Collapse? — AryHHAry · 2026-10-14
- Anthropic researcher reportedly told friend he's 'enjoying my last six months' — notkilleveryoneist · 2026-10-14
- Jan Kulveit Bets Against AI 2027: Reality Tracks My Timeline, Not Kokotajlo's — jankulveit · 2026-10-14
- Charts suggest AI agent flooding is surging complaints and litigation across multiple countries — RexDouglass · 2026-10-14
- The Last AGI Breakthrough May Be a Model's Self-Concept: Knowing Its Decisions Truly Matter — theteknosaur · 2026-10-14
- MIT's Buehler: three weeks before ChatGPT we predicted AI as a new language of science — ProfBuehlerMIT · 2026-10-14