RareDiseaseBench debuts: 35 reconstructed rare disease cases test AI diagnostic agents
kenbwork · x · 2026-11-07
Rare diseases collectively affect 300 million people, and diagnosis often takes years. The authors introduce RareDiseaseBench, a benchmark of 35 diagnostic workups reconstructed from published rare disease cases to test AI agents.
- Each agent starts from the patient's presentation and part of the medical record, then must decide which tests to order, justify them, analyze incoming files, and propose a diagnosis and management plan across one or more visits
- The harness (see figure) gives agents access to data objects like ClinVar and gnomAD; file requests pass through a sidecar judge plus deterministic release checks, and submissions are graded by rubric
- Example cases include a 13-year-old boy with growth restriction, hearing loss, and severe chilblains, where long-read sequencing reveals an MN1 translocation
More from AGI Musings
- Epoch AI Researchers Push for Evidence Transparency in AI R&D Speedup Report Estimating ~2x Acceleration — Jsevillamol · 2026-11-07
- Musk: digital superintelligence will add $25T, boosting global economy 20–30% — XFreeze · 2026-11-07
- ByteDance experiments suggest Full Bandwidth Transformer matches models twice its size — DimitrisPapail · 2026-11-07
- Do you actually know a designer who quit because they couldn't find work? — oykun · 2026-11-07
- Turing Award winner David Patterson: economy output will grow ~100x in 10 years — davidpattersonx · 2026-11-07
- Researcher mocks the 'non-public model access' clout economy — ludwigABAP · 2026-11-07