NEJM AI Study: AI Recommends 3.2x More Costly Workups Than Physicians, Prompts Can Shrink Gap

dvira's team published a paper in NEJM AI that systematically measured a previously overlooked dimension of LLM clinical decision support: the "cost of recommended care." The study gave 24 general-purpose and medical AI systems clinical presentations from 200 real outpatient and emergency cases, asked them to produce assessment and management plans, priced their recommended tests and referrals at Medicare rates, and compared against physicians' written plans.

Confirmed

By default, 20 general-purpose systems recommended care averaging $314 per visit—3.2 times what physicians actually ordered.
In about two-thirds of visits, physicians ordered no tests at all, yet AI models still routinely added a familiar bundle of extra tests.
When physicians reviewed 1,000 plans from 5 systems, 18% of AI plans contained tests absent from the physician plan but consistent with clinical guidelines—meaning over-testing isn't entirely worthless.
Prompt interventions (e.g., instructing models to "consider cost") tested across 19 systems cut the cost ratio substantially to roughly 1.4x, at the cost of reduced diagnostic agreement.

Why it matters

The authors note that existing LLM medical evaluations mostly examine accuracy and safety, while medical decision-making is inherently about trade-offs (e.g., whether to order an MRI or recommend a biopsy); this study is the first to introduce "cost" as a standalone evaluation dimension in top-journal-grade evidence.
The results carry a dual lesson for clinical AI deployment: AI over-testing can inflate healthcare spending, but physicians also miss tests—each has blind spots, so a human-AI complementary review process may be the more realistic path.

2026-10-16 ~ 2026-10-16 · 5 related posts

Primary sources

NEJM AI Paper Adds a Missing Dimension to Medical LLM Benchmarks: Cost of Recommended Care — dvir_a · 2026-10-16 03:54 (UTC+08:00)
24 AI Systems Recommend $314 Per Visit in Care vs Physicians' $97, a 3.2x Gap — dvir_a · 2026-10-16 03:54 (UTC+08:00)
NEJM AI: prompt tuning cuts LLM clinical test-order cost ratio to 1.4x across 19 systems — dvir_a · 2026-10-16 03:54 (UTC+08:00)

[source] NEJM AI Paper Adds a Missing Dimension to Medical LLM Benchmarks: Cost of Recommended Care — dvir_a · 2026-10-16
[source] 24 AI Systems Recommend $314 Per Visit in Care vs Physicians' $97, a 3.2x Gap — dvir_a · 2026-10-16
AI Recommends 3.2x Costlier Care; Models Add Tests in 2/3 of Visits Physicians Ordered None — dvir_a · 2026-10-16
Prompting cuts LLM over-testing costs across 19 systems, but diagnosis agreement drops — dvir_a · 2026-10-16
[source] NEJM AI: prompt tuning cuts LLM clinical test-order cost ratio to 1.4x across 19 systems — dvir_a · 2026-10-16