NEJM AI Paper Adds a Missing Dimension to Medical LLM Benchmarks: Cost of Recommended Care
dvir_a · x · 2026-10-16
The author announces the paper's publication in NEJM AI. Three years ago he and Shelly Shahar began experimenting with LLMs for clinical decision support and felt something was missing in how these models were evaluated: most benchmarks focus on accuracy and safety, while medical decisions involve trade-offs — order an MRI, recommend a biopsy, refer or wait. They added the cost of recommended care as an evaluation dimension.
More from Research
- Google's Willow chip cuts quantum errors 40x, raising Q-Day concerns for Bitcoin — jamestagg · 2026-10-16
- A Cookbook for Machine Learning: compiling ML goals into tractable optimization — goyal__pramod · 2026-10-16
- Transluce vets: a year of neuron autointerp work still can't explain complex behaviors — aryaman2020 · 2026-10-16
- Transformer trained on flying bat recordings yields hippocampus 'digital twin' revealing hidden structure — patrickmineault · 2026-10-16
- One year of Recursive Language Models: GPT-5-mini beats GPT-5 by 110% on 132k-token tasks, cheaper too — a1zhang · 2026-10-16
- MIT PhD Sharut Gupta joins DeepMind; Isola hails her representational alignment defense talk — phillip_isola · 2026-10-16