Aggregate WER hid the real problem: half of bank code reads were wrong in production voice agent
Adventurous_Whole973 · reddit · 2026-09-21
A voice-agent developer shares a production lesson: their ASR showed 95% accuracy, yet manual checks found nearly half of customer-read IFSC bank codes had at least one wrong character.
Key failure modes:
- On 8kHz phone audio with Indian accents, M/N, S/F, B/D/P are easily confused
- Digits were worse: "double five", "fifty five" and "five five" were treated as three different inputs
- Uncommon surnames and mixed alphanumeric policy numbers were basically random
- About half of callers code-switch between Hindi and English mid-sentence, and the model commits to one language, breaking exactly when the critical info is read
The takeaway: a single aggregate WER told the collections team nothing — a wrong IFSC means a failed payment, a wrong name means a compliance flag. They now track per-field accuracy and are exploring constrained decoding and validation passes, though unsure if that just patches a model-level problem.
More from coding & agent
- PufferLib author rants about LLM-generated contributor code, semi-automates fixes with a Skill file — jsuarez · 2026-09-21
- Alsop slams Claude and Codex subscription shrinkflation, routes tasks via classifier to cut tokens — StewartalsopIII · 2026-09-21
- Building a homework-correction pipeline in 30 minutes with Ling-3.0-flash-VL — nikola_mr64990 · 2026-09-21
- Ant's new Ling-3.0-flash-VL turns a design screenshot into working HTML+CSS in one shot — nikola_mr64990 · 2026-09-21
- 10-second screen recording in, working website out: a hands-on test of Ling-3.0-flash-VL's video-to-code — nikola_mr64990 · 2026-09-21
- Devs Debate AI as a Multiplier: Weak Fundamentals Mean Faster Tech Debt and Security Holes — RachelVT42 · 2026-09-21