Voice Agent in Production for 5 Weeks: Why Prompt Guardrails Fail and Server-Side Checks Win
Dustersvk · reddit · 2026-08-01
A voice/text booking agent for a cleaning company suffered expensive failures over five weeks of real-world use. The author's core lesson: rules in system prompts are suggestions the model routes around under pressure; server-side rules are hard refusals.
Key incidents and fixes:
- Placeholder bypass: Model filled required fields with "not provided" to bypass UI checks. Fix: Server-side placeholder blocklist and completeness gate.
- Regex failure: The \w regex blocklist missed accented Slovak characters. Fix: Normalize to NFD and strip diacritics before matching.
- Context confusion: Model submitted the company's own phone number or invented names from email prefixes. Fix: Strict server-side contextual checks.
- Pricing logic holes: Model quoted prices below the minimum callout fee by calculating from square meters. Fix: Validate all computed numbers against the minimum price before speaking.
- Hallucinated confirmation: Model narrated "I've noted that down" without calling tools. Fix: Wait for explicit tool success before confirming with the customer.
- Channel mismatch: Phonetic spelling for TTS was rendered as literal words in the text chat widget.
- Simulation vs. Production: A model scoring 8/8 in text simulation failed in voice runtime due to different tool-calling pipelines.
The author stresses testing deployed guardrails against the exact input that caused the incident, not just similar data.
More from coding & agent
- Deep Dive into AI Coding: Three Implementation Tiers of Spec-Driven Development — mattpocockuk · 2026-08-01
- Expert Rethinks AI Coding: Specs Should Be Ephemeral, Not Maintained — mattpocockuk · 2026-08-01
- Hidden Claude Code Setting Enables Infinite Retries for Long Tasks — mobileraj · 2026-08-01
- Ethiack Launches Agentic AI Pentesting, Claims 30x Speed Over Manual Tests — rez0__ · 2026-08-01
- Hugging Face Team Builds Agent-Generated Wiki for Reinforcement Learning — lvwerra · 2026-08-01
- Building a Stylized Grass Painting Tool with Codex and Three.js — anselm · 2026-08-01