HealthAdminBench: Claude Opus 4.6 completes just 36.3% of admin workflows end to end
jonc101x · x · 2026-10-28
Part 8/14 of Stanford's medical AI benchmark thread: on HealthAdminBench, Claude Opus 4.6 completed just 36.3% of administrative workflows end to end, while GPT-5.4 succeeded on 82.8% of individual subtasks but finished only 26.7% of entire workflows. The cited tweet notes the dominant failure mode: agents correctly document the plan but never place the order in the EHR.
More from coding & agent
- Managing multiple coding agents: hourly-granularity tasks and e2e-only acceptance — dotey · 2026-10-28
- Supercharging LLM classification with logprobs: soft classifiers you can threshold-tune — JnBrymn · 2026-10-28
- Codex daily build series day 23: skipping iCloud plugins for a reset — dotey · 2026-10-28
- Cloudflare Access adds fully customizable pages with HTML + Liquid templates — dinasaur_404 · 2026-10-28
- Opus retried every session 20.6 million times during a memory leak hunt — johnlindquist · 2026-10-28
- Open-Source Skill Teaches AI Agents to Design and Produce Videos — songguoxiansen · 2026-10-28