Root Cause of LLM Medical Triage Errors: Misaligned Utility, Not Lack of Knowledge

erichorvitz · x · 2026-08-04

Recent research reveals that when LLMs give poor medical triage recommendations, the root cause is an implicitly misaligned utility function rather than a lack of medical knowledge—the models actually know quite well which cases are more severe.

The study introduces decision-analytic steering. In high-stakes decisions like emergency triage, different errors carry vastly different consequences (e.g., missing a critical patient is far costlier than an unnecessary ER visit). Thus, a system making recommendations under uncertainty must do more than predict the likely outcome; it must weigh the relative costs and benefits of acting or failing to act.

Original post →

Root Cause of LLM Medical Triage Errors: Misaligned Utility, Not Lack of Knowledge

More from Safety