12 open speech models tested on 208 drug names: best got 169 right, 16 stumped all
MaziyarPanahi · x · 2026-11-17
MaziyarPanahi tested 12 open speech recognition models on 208 drug names drawn from PriMock57 (57 mock GP consultations, CC BY 4.0). The spread was huge: the best model, Nemotron 3.5 Transcribe, got 169 right; Whisper large-v3 followed at 75% and Qwen3-ASR 1.7B at 74%. The worst managed only 104, and 16 drug names were missed by every model — lisinopril came out as "listen april." His takeaway: a medical AI scribe is only as safe as its ears.
More from Models
- "I don't even know why I'm using fable-5.2": dev admits model FOMO keeps him hooked — adonis_singh · 2026-11-17
- Gemini 4 Carbon cache hit rate averages 85.2%, raising cost-efficiency doubts — mariofilhoml · 2026-11-17
- Claude Code Power User Burns 53% of Weekly Limit in Two Days, Downgrades to Sonnet — Inevitable_Truth_85 · 2026-11-17
- SenseNova Loop 9B Claimed to Match Qwen3 27B at 9B Params, GGUF Coming — MeinDruckerSpinnt · 2026-11-17
- SenseNova-Loop-9B Lands on Hugging Face, Claiming 27B-Class Performance — Marcuss2 · 2026-11-17
- AA-PersonalityBench ranks Gemini 4 Argon most metaphorical model, GPT-3.5 Turbo least — ArtificialAnlys · 2026-11-17