Clinical trials already hedge against human reward hacking, says Eisen on Tao's AI drug concern
tallinzen · x · 2026-10-12
Responding to Terence Tao's concern about AI reward hacking in drug discovery, Michael Eisen argues it's not new: clinical trials are structurally a hedge against reward hacking by humans and corporations — and are routinely gamed, though by corrupting the process rather than an AI designing something that merely looks effective. He thinks Tao's worry is valid but the example misses the point. tallinzen adds the slide is clearly about alignment/reward hacking, not using drugs without understanding mechanisms.
Related event: AI reward hacking in drug trials sparks debate(4 posts)→
More from AGI Musings
- From lathe to prompt: the Depression-era basement inventor hustle lives on in AI — sull · 2026-10-12
- DeepMind's Nando de Freitas recommends a thoughtful new Hinton interview — NandoDF · 2026-10-12
- Palantir CTO Shyam Sankar: doomers have been wrong about literally everything — eliano · 2026-10-12
- Research: AI Content Flood Is Already Eroding Human Creative Innovation — AlexTensor · 2026-10-12
- WaPo: In the AI dominance race, the US relies on a surprising ally — China — yogthos · 2026-10-12
- IonQ roadmap could break Bitcoin encryption by 2028, leaving migration too late — jamestagg · 2026-10-12