Nature Medicine Matters Arising disputes claims that general LLMs beat clinical AI tools
anshulkundaje · x · 2026-09-03
A widely shared study claimed general-purpose LLMs outperform specialized clinical AI tools like Open Evidence and UpToDate.
In a Matters Arising published in Nature Medicine, beaulieujones and colleagues argue that the conclusions many have drawn from that work are not substantiated by the evidence presented, urging caution about sweeping claims that general LLMs beat purpose-built clinical AI systems.
More from Research
- LLMs are bad at Puzzlescript games, PuzzleJax team tells IEEE CoG — Amidos2006 · 2026-09-03
- ICM panel with Akshay Venkatesh and top mathematicians on AI and math now out — AlexKontorovich · 2026-09-03
- Philosopher says Fable 5.1 found Arrhenius's favored population ethics impossibility theorem is false — willmacaskill · 2026-09-03
- AI Is Forcing Mathematicians to Ask What They Really Value About Math — stevenstrogatz · 2026-09-03
- Caltech and NVIDIA Show Physics-Aware AI for Extreme Weather Prediction — AnimaAnandkumar · 2026-09-03
- Agents transmit invented languages to newcomers, enabling cumulative cultural evolution — EliasEskin · 2026-09-03