AI-for-science benchmarking markets: single evals may cost months and millions, a durable niche
kenbwork · x · 2026-10-30
kenbwork argues model capabilities in science have no obvious ceiling, and economic utility keeps rising as impact jumps from incremental research steps to consumer-legible outcomes like cures. Single evaluations may take months, millions of dollars and world-class teams, making benchmarking one of the few durable components of the AI x science ecosystem. As these markets form with unknown growth potential, he says, use your imagination.
More from AGI Musings
- Carmack: AI writing software from scratch isn't optimal — well-regularized building blocks matter — ID_AA_Carmack · 2026-10-30
- Economist Walks Through 'Unserious AI Use' in a Published Saez-Zucman-Adjacent Paper — anshulkundaje · 2026-10-30
- 'Agents Are the New Buyers': User Solves App Problem Without Knowing It Runs on Supabase — gabe_ragland · 2026-10-30
- "If everyone's convinced AI is a bubble, there's a good chance it isn't" — justalexoki · 2026-10-30
- Altman says local AI wastes scarce memory; critics say he misses the point — kimmonismus · 2026-10-30
- Assistance-games framing: models should ask questions and accept correction — dhadfieldmenell · 2026-10-30