Hamel Husain: Hard-to-eval products are bad products — and AI makes data science more valuable
hugobowne · x · 2026-09-05
Hugo Bowne-Anderson releases the podcast episode The Rise of the AI Scientist with Hamel Husain.
- Core argument: An AI data agent tells you net revenue without showing metric definitions, source tables, queries, or assumptions — and you can't trust the answer. Husain frames this as a product design failure: "If a product is hard to eval, it's a strong smell the product isn't good." The fix is exposing the evidence and checks a domain expert actually uses, not bolting on another scoring pipeline.
- Data science revival: Agents generate more data, noisier signals, and plausible-looking outputs nobody knows whether to trust. "AI has made data science way more valuable than ever," and the people doing this reasoning and debugging may eventually be called AI scientists.
- The episode runs 1h10m and also promotes the rebuilt AI Evals for Engineers & PMs course by Husain and Shreya Shankar.
More from AGI Musings
- a16z: Superhuman AI hardware design is the easy part — manufacturing is the hard one — a16z · 2026-10-21
- a16z's Casado says AI can't do a VC's job — but it can do everything around it — alvelda · 2026-10-21
- Kent Beck: Let the Genie Carry the Toil, But Keep the Struggle — josh_wills · 2026-10-21
- Nearly $3B mapped: the money behind the AI existential risk ecosystem — KeyNaive2982 · 2026-10-21
- Three reasons AI is not conscious and may never be, per Nature essay — anilkseth · 2026-10-21
- Everything in AI is downstream of a global status graph, says founder — gabriel1 · 2026-10-21