12 open speech models tested on 208 drug names: best got only 169 right

MaziyarPanahi · x · 2026-11-17

Developer Maziyar Panahi benchmarked 12 open speech models on 208 drug names drawn from 57 mock GP calls — same clips, cut at human transcript sentence boundaries, no hints, one GPU, and a name only counted if spelled the way a pharmacist reads it.

The spread was huge:

Takeaway: a medical scribe is only as safe as its ears — transcription errors are a real safety risk in medication contexts.

Related event: Test of 12 open speech models on 208 drug names finds best gets only 169 right(3 posts)→

Original post →

More from Models

Models channel →