Study: AI beat physicians in 1,100 specialist consult cases
HealthcareAIGuy · x · 2026-07-21
A new ARISE lab-led study evaluated 45 AI models and clinical tools across 1,100 specialist consult cases.
Key takeaways:
- AI alone outperformed physicians using conventional resources or AI; the post argues that if a PCP simply followed the AI’s recommendation, patients would likely benefit, though more validation is still needed.
- Specialized clinical platforms such as AMBOSS, Doximity, OpenEvidence, and Glass Health outperformed general-purpose LLMs.
Related event: AI Outperforms Doctors in Specialty Consultations(3 posts)→
More from Research
- InterviewPlayground: JHU's simulated eval for AI interviewers hits 0.86 correlation — anjalie_f · 2026-10-27
- Formal methods expert: never trust LLM-generated specs, vet them instead — davidmanheim · 2026-10-27
- Reasoning models can barely control their chain of thought, paper finds — tomekkorbak · 2026-10-27
- New paper: LLM data schedules and learning rate schedules should be co-designed — jasondeanlee · 2026-10-27
- What makes research get adopted: important problems, proper evals, and usability — srchvrs · 2026-10-27
- First-week PhD student lands ARR Area Chair role, exposing peer-review quirks — anmarasovic · 2026-10-27