12 open speech models tested on 208 drug names: even the best gets 27 wrong
MaziyarPanahi · x · 2026-11-17
Maziyar Panahi benchmarked 12 open speech models on 208 drug names from 57 mock GP calls. The spread is huge: the best, Nemotron 3.5 Transcribe, got 169 right; the worst only 104. Sixteen drug names were missed by every model — lisinopril famously came out as "listen april". His takeaway: a medical scribe is only as safe as its ears.
More from Models
- Counterfactual benchmark shows 11 frontier models fail to grasp programming semantics — jessyjli · 2026-11-17
- Harness Adapters: RL-trained LoRA lifts Qwen3.8-27B on Pi from 29% to 63% on SWE-bench Pro — LysandreJik · 2026-11-17
- Qwen says offensive-cyber training data was removed — placebo or real lobotomy? — Rare_Subject2343 · 2026-11-17
- d1-Omni proves a strict VLM judge, rejecting bboxes accepted by Gemma and LFM2.5-VL — mervenoyann · 2026-11-17
- KORMo-Web-11B, a Korean VLM for GUI and web agents, trends on Hugging Face — KORMo-VL · 2026-11-17
- Uncensored Qwen3.5-4B aggressive fine-tune trends on Hugging Face — HauhauCS · 2026-11-17