12 open speech models tested on 208 drug names: best got 169 right, 16 stumped all

MaziyarPanahi · x · 2026-11-17

MaziyarPanahi tested 12 open speech recognition models on 208 drug names drawn from PriMock57 (57 mock GP consultations, CC BY 4.0). The spread was huge: the best model, Nemotron 3.5 Transcribe, got 169 right; Whisper large-v3 followed at 75% and Qwen3-ASR 1.7B at 74%. The worst managed only 104, and 16 drug names were missed by every model — lisinopril came out as "listen april." His takeaway: a medical AI scribe is only as safe as its ears.

Related event: Test of 12 open speech models on 208 drug names finds best gets only 169 right(3 posts)→

Original post →

More from Models

Models channel →