FDA's GenAI medical device paper exposes the accountability gap in AI clinical practice
jonc101x · x · 2026-09-09
A physician's thread on AI in clinical medicine, anchored by the FDA's new discussion paper:
- Capability layers: LLMs already dominate written multiple-choice medical exams, but lag on reasoning through dynamic cases under uncertainty and on clinical skills like history taking, physical exams, and communication.
- Accountability gap: Duty, dereliction, causation, and damages have no good human-computer analogy—if an autonomous AI harms a patient, who is the defendant, who sits for deposition, who pays damages?
- Medical training analogy: Even acing exams and simulations, trainees still need institutional vouching plus ongoing oversight (license renewal, complaints, malpractice suits)—addressing the "smart sociopath" problem.
- FDA paper: The Digital Health Center of Excellence released "Considerations for the Regulation of Generative AI-Enabled Medical Devices" with a request for feedback, acknowledging that frameworks built for static devices don't work for constantly evolving generative systems, covering risk assessment, premarket evaluation, and postmarket monitoring.
More from AGI Musings
- Stanford-Harvard ARISE releases inaugural State of Clinical AI Report 2026 — jonc101x · 2026-09-09
- Anthropic researcher: without AI, US GDP growth would be ~1% — QuintinPope5 · 2026-09-09
- Ben Todd: if alignment experts say it could kill everyone, react with "oh shit", not PR critiques — sjgadler · 2026-09-09
- Is 'Pointing Enough ChatGPT Agents at Navier-Stokes' an Extreme Idea? — teortaxesTex · 2026-09-09
- "Change Could Come Any Day": Poster Distrusts OpenAI and Anthropic on Alignment — justalexoki · 2026-09-09
- tszzl: solving mechinterp within a year could halve his p(doom); utopia is seriously possible — tszzl · 2026-09-09