Expert panels map 21 clinical AI risks; adversarial tests show autonomous care still unsafe
AleksandraFaust · x · 2026-10-31
Valentin Lievin and colleagues ran two Delphi panels (30 experts) to define consensus taxonomies for AI in clinical settings: a clinical safety taxonomy (6 domains, 21 risks — spanning acute presentation, medication safety, diagnostic scope, mental health, system exploitation including prompt injection, and autonomy/safety-netting) and a relational competency framework (5 domains, 18 items, covering human-AI relational awareness, crisis recognition, clinical competency, cultural sensitivity, and scope management).
Stress-testing on 256 taxonomy-derived adversarial scenarios (concealed crises, pressure for unsafe prescriptions) showed training nearly eliminated contraindication violations and more than halved acute under-triage. Far fewer failures — but still too many for autonomous care today, the authors conclude.
More from Safety
- Anthropic adds invisible text watermark to Claude outputs starting Nov 3 — petesena · 2026-10-31
- LessWrong essay: AI lab employees are 'microdosing' their org's spirit, argues researcher — jankulveit · 2026-10-31
- Stanford prof publishes lab AI policy: no AI-authored text in papers, ever — ChrisGPotts · 2026-10-31
- Security researcher: air-gapped AI agents are only possible with open-weight models — wunderwuzzi23 · 2026-10-31
- Podcast: Is Mistral Large 4 the West's best open model? Plus AI cyber threats — The Cognitive Revolution · 2026-10-31
- 80,000 Hours podcast: Coefficient Giving's Luke Muehlhauser on AI funding strategy and timelines — lxrjl · 2026-10-31