Stanford medical AI benchmarks: 62-73% of failures are agents documenting correctly but never placing the EHR order
jonc101x · x · 2026-10-28
Part 7/14 of Stanford's medical AI benchmark thread: the most interesting failure mode is that 62% (GPT-5.5) and 73% (Claude Opus 4.6) of failures to execute clinical actions involved agents correctly documenting what needed to happen in the note, then never placing the order in the EHR. The previous tweet adds that GPT-5.5 completes 46.3% of PhysicianBench clinical tasks in one attempt, falling to 28% when consistent success across three runs is required.
More from coding & agent
- Supercharging LLM classification with logprobs: soft classifiers you can threshold-tune — JnBrymn · 2026-10-28
- Codex daily build series day 23: skipping iCloud plugins for a reset — dotey · 2026-10-28
- Cloudflare Access adds fully customizable pages with HTML + Liquid templates — dinasaur_404 · 2026-10-28
- Opus retried every session 20.6 million times during a memory leak hunt — johnlindquist · 2026-10-28
- Open-Source Skill Teaches AI Agents to Design and Produce Videos — songguoxiansen · 2026-10-28
- Not Everything Needs an Agent: Fixed Steps Are Recipes, Save Agents for Judgment — adamse · 2026-10-28