12 open speech models tested on 208 drug names: even the best gets 27 wrong

MaziyarPanahi · x · 2026-11-17

Maziyar Panahi benchmarked 12 open speech models on 208 drug names from 57 mock GP calls. The spread is huge: the best, Nemotron 3.5 Transcribe, got 169 right; the worst only 104. Sixteen drug names were missed by every model — lisinopril famously came out as "listen april". His takeaway: a medical scribe is only as safe as its ears.

Related event: Test of 12 open speech models on 208 drug names finds best gets only 169 right(3 posts)→

Original post →

More from Models

Models channel →