Voice agent hallucinated a delivery date and defended it for three turns — until context was split
Street-Support4149 · reddit · 2026-10-10
A voice-agent developer shares a real incident where the model treated its own words as facts.
- The agent invented a delivery date early in a call, then doubled down as customers pushed back: its own words sat in context and were treated as ground truth. It built a whole plan around the fake date and even apologized for delays. It sounded most confident when repeating itself, and reviewers only audited the first minute — the only part that worked.
- Prompting it to treat its own answers as provisional failed after a few turns, like every long-context instruction.
- What worked: splitting context into what the customer said, what a tool returned, and what the agent said — only the first two count as facts now.
- Trade-off: the agent sometimes audibly contradicts itself when tool results conflict, but it's at least correct. Call scores dipped for weeks post-launch. The author notes Gnani and others separate tool state from conversation context at the platform level.
More from coding & agent
- Claude Code Pushes Urgent Security Update Across All Installations — umeshonai · 2026-10-23
- Claude cut a 2.6-hour video export to 20 minutes — but only after being told to rebuild the server — byhewar · 2026-10-23
- 3 flight-booking rules my AI agent checks that I'd never do by hand — Efistoffeles · 2026-10-23
- OpenJet claims Qwen3.8-27B coding runs 1.9x faster than llama.cpp on one RTX 3090 — JeremyCMorgan · 2026-10-23
- Building an autonomous SEO agent in Hermes with three skills: programmatic pages, Keywords Everywhere, DataForSEO — EXM7777 · 2026-10-23
- Building a sports betting AI agent with onchain execution and human approval by default: architecture and lessons — Dry-Comfortable-2678 · 2026-10-23