gregmushen on why AI's exponential progress can't bypass biology's empirical bottlenecks
gregmushen · x · 2026-10-12
In a lengthy X debate with @reallyoptimized, gregmushen lays out a three-question framework for why AI won't simply shortcut medical breakthroughs:
- Can a model discover the correct answer?
- Is there enough information in the inputs to distinguish the correct answer?
- Can we verify the answer is correct?
His core argument: people correctly predict #1 (exponential model progress) but overlook that #1 remains subject to #2 and #3, which don't inherently benefit from technology S-curves — leading to overestimated impact and underestimated timelines.
Citing NHANES, BIOBANK and thousands of RCTs, he argues existing observational data can find correlations but cannot establish causal effects of novel interventions: if ASI invented a cellular-reprogramming therapy tomorrow, the number of people in any dataset who received it would be exactly zero. His interlocutor counters that new, currently incomprehensible evidentiary standards will emerge and biology may become an information technology; gregmushen insists any replacement standard itself requires empirical evaluation, and progressive human-subject testing remains unavoidable.
More from AGI Musings
- From CSS Zen Garden to headless internet: HCI is becoming HHI — signulll · 2026-10-13
- Amazon now has over 1 million robot workers. What's next for human employees? — lulzxdxdxd · 2026-10-13
- Claymation won't die to AI: the process matters as much as the product — dylfreed · 2026-10-13
- Philosopher Carissa Véliz on how automatic AI translation erases language culture — CarissaVeliz · 2026-10-13
- Every's Thesis: 2027 conference to explore human work after automation — every · 2026-10-13
- Grady Booch: coding agents merely pattern-match and can't do real architecture — Grady_Booch · 2026-10-13