Prompting cuts LLM over-testing costs across 19 systems, but diagnosis agreement drops
dvir_a · x · 2026-10-16
Researcher analyzing LLM care plans' over-testing problem: in two-thirds of visits physicians ordered no tests, yet models often added familiar panels anyway. However, a physician review of 1,000 plans from five systems found 18% included guideline-concordant workup the physician had missed.
Prompt interventions: asking 19 systems to be cost-conscious lowered the cost ratio to 2.5×; asking them to focus on management-changing decisions while considering dangerous diagnoses brought it to 1.4× — but reduced agreement with the physician's diagnosis in 17 of 19 systems. The author notes this doesn't prove worse care (patient outcomes weren't measured), but shows we must evaluate what else changes when optimizing one dimension.
More from Research
- Google's Willow chip cuts quantum errors 40x, raising Q-Day concerns for Bitcoin — jamestagg · 2026-10-16
- A Cookbook for Machine Learning: compiling ML goals into tractable optimization — goyal__pramod · 2026-10-16
- Transluce vets: a year of neuron autointerp work still can't explain complex behaviors — aryaman2020 · 2026-10-16
- Transformer trained on flying bat recordings yields hippocampus 'digital twin' revealing hidden structure — patrickmineault · 2026-10-16
- One year of Recursive Language Models: GPT-5-mini beats GPT-5 by 110% on 132k-token tasks, cheaper too — a1zhang · 2026-10-16
- MIT PhD Sharut Gupta joins DeepMind; Isola hails her representational alignment defense talk — phillip_isola · 2026-10-16