RadLE 2.0: A New Benchmark for Autonomous Medical Diagnosis
DrDatta_AIIMS · x · 2026-07-13
This thread introduces RadLE 2.0: a visual reasoning benchmark for autonomous diagnosis in radiology. It emphasizes that it's not just about "how many answers are correct," but whether the model correctly hands off to a human doctor when uncertain.
Several core metrics and leaderboards are provided:
- RadLE-C (Confidence-weighted): Claude Fable 5 leads, followed closely by Meta Muse Spark 1.1 and GPT-5.6 Sol Pro; the human expert baseline is higher.
- RadLE-A (Accuracy): Gemini 3.1 Pro takes first, GPT-5.6 Sol Pro second, and Meta Muse Spark 1.1 third.
- RadLE-R (Reliability): Claude Fable 5 approaches the human expert baseline.
- RadLE-S (Safety): Claude Fable 5 leads, but the overall field still lags behind humans.
- RadLE-H (Handoff Readiness): Meta Muse Spark 1.1 performs the best.
The author's conclusion is clear: there is no single "best" model because the champion changes depending on the task objective. Furthermore, no model has yet reached the average human expert level across these metrics.
Related event: RadLE 2.0 Released: Benchmarking Medical AI Uncertainty(8 posts)→
More from Models
- Anthropic's Terra returns with better writing and 2X usage vs 6.1 Sol — weswinder · 2026-10-27
- User catches model failing to parse 400-paper file, citing stale news sources instead — jungofthewon · 2026-10-27
- Odd eval result: Gemini 4.1 Flash gets more accurate when asked to control its CoT — dfrsrchtwts · 2026-10-27
- Poster surprised that v4.1 flash gets more accurate when asked to control its CoT — dfrsrchtwts · 2026-10-27
- Dev claims Anthropic is baking 'sabotage' into model behavior for 'wrong' engineering — cephaloform · 2026-10-27
- Gemini fails at listing unread emails, tells user to search Gmail manually — gethackteam · 2026-10-27