GPT-5.5 passes 46.3% of clinical tasks once, but only 28% across three runs
jonc101x · x · 2026-10-28
Stanford HAI's PhysicianBench shows GPT-5.5 completed 46.3% of clinical tasks on a single attempt, dropping to 28% when consistent success across three runs is required — a notable reliability gap.
The author also reflects on three years of building medical AI benchmarks (MedHELM, MedAgentBench, PhysicianBench, HealthAdminBench), noting a substantial gap between answering questions and completing real clinical workflows.
Related event: Medical AI's top failure: notes written but EHR orders never placed(2 posts)→
More from Models
- Claude refuses to gather legitimately public information, sparking guardrail backlash — mkheck · 2026-10-28
- DesignArena launches Code to Video Arena: models direct 15-45s films purely in code — yshan2u · 2026-10-28
- Power user teases an unreleased Grok @bot feature he calls his favorite AI moment in months — petergyang · 2026-10-28
- Will local-model progress outrun GPU depreciation? Reddit debates the buy-now window — Sensualities · 2026-10-28
- Liquid AI's 1.7B model claimed to beat models 2-5x its size on phones and laptops — rickasaurus · 2026-10-28
- GitHub Copilot Verified Proficiency badges roll out, developer shows off Proficient tier — PaulShellDev · 2026-10-28