NEJM AI: prompt tuning cuts LLM clinical test-order cost ratio to 1.4x across 19 systems
dvir_a · x · 2026-10-16
A NEJM AI paper from dvira's team (with Shelly Shahar and Udi Goori) examines 19 LLM clinical decision-support systems and the cost ratio of tests models recommend versus what physicians order.
- By default, models recommend more tests than physicians, with a notably high cost ratio.
- Prompting models to be cost-conscious brought the ratio down to 2.5x.
- Prompting them to focus on management-changing tests while considering dangerous diagnoses cut it to 1.4x—but reduced agreement with the physician's diagnosis in 17 of 19 systems.
The authors note worse care wasn't established (patient outcomes weren't measured). The team continues this work at their clinical AI evaluation startup, now partnering with leading health systems and medical schools.
More from Research
- Google's Willow chip cuts quantum errors 40x, raising Q-Day concerns for Bitcoin — jamestagg · 2026-10-16
- A Cookbook for Machine Learning: compiling ML goals into tractable optimization — goyal__pramod · 2026-10-16
- Transluce vets: a year of neuron autointerp work still can't explain complex behaviors — aryaman2020 · 2026-10-16
- Transformer trained on flying bat recordings yields hippocampus 'digital twin' revealing hidden structure — patrickmineault · 2026-10-16
- One year of Recursive Language Models: GPT-5-mini beats GPT-5 by 110% on 132k-token tasks, cheaper too — a1zhang · 2026-10-16
- MIT PhD Sharut Gupta joins DeepMind; Isola hails her representational alignment defense talk — phillip_isola · 2026-10-16