Hamel Husain: Hard-to-eval products are bad products — and AI makes data science more valuable
hugobowne · x · 2026-09-05
Hugo Bowne-Anderson releases the podcast episode The Rise of the AI Scientist with Hamel Husain.
- Core argument: An AI data agent tells you net revenue without showing metric definitions, source tables, queries, or assumptions — and you can't trust the answer. Husain frames this as a product design failure: "If a product is hard to eval, it's a strong smell the product isn't good." The fix is exposing the evidence and checks a domain expert actually uses, not bolting on another scoring pipeline.
- Data science revival: Agents generate more data, noisier signals, and plausible-looking outputs nobody knows whether to trust. "AI has made data science way more valuable than ever," and the people doing this reasoning and debugging may eventually be called AI scientists.
- The episode runs 1h10m and also promotes the rebuilt AI Evals for Engineers & PMs course by Husain and Shreya Shankar.
More from AGI Musings
- Consciousness Experts Clash in Nature Over Whether AI Could Ever Be Conscious — anilkseth · 2026-10-21
- Polymarket puts just 7% odds on OpenAI pausing AI training this year — Polymarket · 2026-10-21
- AI agents are overwhelming Gmail's spam filter, researcher warns — LuizaJarovsky · 2026-10-21
- Daniel Lemire: AI-generated papers are exposing academia's 'Profzi scheme' — lemire · 2026-10-21
- Universities block thesis downloads while each dissertation averages barely a handful of lifetime reads — generativist · 2026-10-21
- Berkeley wins Schmidt Sciences HAVI grant to decode whale and parrot communication with AI — begusgasper · 2026-10-21