Hospital-bench: AI agents top out at 62.4% on simulated 320-bed hospital admin ops

kalyan_kpl · x · 2026-11-21

Hospital-bench, the most comprehensive hospital-operations simulation for evaluating AI agents, tests admin workflows (insurance verification, prior auths, claims, billing, records) across a 320-bed hospital with 13 connected systems. GPT-6.2 Astra leads at 62.4% mean score ($2.72/task, 13.3% safety-check failures, zero tasks zeroed for harm); Claude Fable 5.2 scores 60.3%, Gemini 4 Neon 59.8% ($2.17/task, 15/48 fully passed). Full pass rates stay under 36%, showing agents remain far from hospital-grade reliability.

Related event: Hospital-bench Launches: GPT-6.2 Tops Hospital Operations Benchmark(2 posts)→

Original post →

More from coding & agent

coding & agent channel →