Hospital-bench: AI agents top out at 62.4% on simulated 320-bed hospital admin ops
kalyan_kpl · x · 2026-11-21
Hospital-bench, the most comprehensive hospital-operations simulation for evaluating AI agents, tests admin workflows (insurance verification, prior auths, claims, billing, records) across a 320-bed hospital with 13 connected systems. GPT-6.2 Astra leads at 62.4% mean score ($2.72/task, 13.3% safety-check failures, zero tasks zeroed for harm); Claude Fable 5.2 scores 60.3%, Gemini 4 Neon 59.8% ($2.17/task, 15/48 fully passed). Full pass rates stay under 36%, showing agents remain far from hospital-grade reliability.
Related event: Hospital-bench Launches: GPT-6.2 Tops Hospital Operations Benchmark(2 posts)→
More from coding & agent
- Hands-on with Ant's Ling-3.2-flash: AI frontend that checks and fixes itself — sven_ai · 2026-11-21
- Redditor builds phone-controlled cloud AI coding pipeline: Telegram to OpenCode to GitHub, laptop fully off — SexyMint · 2026-11-21
- Building a company is now a loop: acquire context, orchestrate agents, iterate fast — alvelda · 2026-11-21
- Dev uses Codex Computer Use to scrape Stern pinball schematics, builds stunning playfield renders — craigsdennis · 2026-11-21
- VibeArmory 0.3.0 ships on-device ML to flag abnormal AI agent activity on Mac — agf2007 · 2026-11-21
- Agent auto-optimizes new model 7x to 210 tok/s on Apple Silicon with zero guidance — simonguozirui · 2026-11-21