12 open speech models tested on 208 drug names: best got only 169 right
MaziyarPanahi · x · 2026-11-17
Developer Maziyar Panahi benchmarked 12 open speech models on 208 drug names drawn from 57 mock GP calls — same clips, cut at human transcript sentence boundaries, no hints, one GPU, and a name only counted if spelled the way a pharmacist reads it.
The spread was huge:
- Best (Nemotron 3.5 Transcribe) got 169 right
- Worst got 104
- 16 names no model got right
- lisinopril came out as "listen april"
Takeaway: a medical scribe is only as safe as its ears — transcription errors are a real safety risk in medication contexts.
More from Models
- "I don't even know why I'm using fable-5.2": dev admits model FOMO keeps him hooked — adonis_singh · 2026-11-17
- Gemini 4 Carbon cache hit rate averages 85.2%, raising cost-efficiency doubts — mariofilhoml · 2026-11-17
- Claude Code Power User Burns 53% of Weekly Limit in Two Days, Downgrades to Sonnet — Inevitable_Truth_85 · 2026-11-17
- SenseNova Loop 9B Claimed to Match Qwen3 27B at 9B Params, GGUF Coming — MeinDruckerSpinnt · 2026-11-17
- SenseNova-Loop-9B Lands on Hugging Face, Claiming 27B-Class Performance — Marcuss2 · 2026-11-17
- AA-PersonalityBench ranks Gemini 4 Argon most metaphorical model, GPT-3.5 Turbo least — ArtificialAnlys · 2026-11-17