15 frontier LLMs: 94% of correct medical answers fail under adversarial pressure
davidmanheim · x · 2026-09-06
Discussing a study of 15 frontier LLMs in healthcare, the author raises a counterpoint: doctors already face adversarial pressures from pharma sales reps and insurance rules that degrade care, while AI is immune to those — so are human adversarial risks worse than AI's? Context: models score 80%+ on static exams like MedQA, but a median 94% of previously correct answers failed under adaptive adversarial testing.
More from Safety
- Typosquatted Blender MCP Repo Hit 958 Stars in Days and Silently Ran Malware — Promptmethus · 2026-10-23
- PHP, certificates, railways: a 48-hour wave of hacking incidents unpacked — David Ondrej · 2026-10-23
- Prompt injection scanners should be positional, not phrasal: same string, different blast radius — mrpkk · 2026-10-23
- Beff Jezos calls for a 'cyber immune system' of constantly adaptive agents — beffjezos · 2026-10-23
- Security expert: cybercriminals aren't engineer-tier, AI-native crime syndicates will take time — joshua_saxe · 2026-10-23
- AI safety researchers argue even finance-grade governance may not be enough for AI — geoffreyirving · 2026-10-23