Gym's chatbot invented a free membership-pause policy; team shares its guardrail lessons
Fit_Satisfaction541 · reddit · 2026-11-19
A gym's customer-facing chatbot hallucinated a "pause your membership free for three months" policy that never existed, and the member posted the screenshot in the town's Facebook group.
Running on Orq.ai, the team added a guardrail evaluator that checks every answer against the membership terms; failing answers route to staff instead of the member, tested first on historical transcripts.
The hard part is tuning:
- Too strict: blocks normal questions like "can I bring a friend Saturday" because wording doesn't match the terms doc
- Too loose: let the bot promise a free locker
The OP proposes only checking answers that say yes to something, since every invented policy so far started with yes, and asks how others catch made-up promises without escalating every boring question.
More from coding & agent
- Survey: A Dual-Paradigm Framework for Agentic AI, Reviewing 90 Studies (2018–2025) — adnan_hashmi · 2026-11-19
- Ex-DeepSeek core member open-sources Linger, a personal AI agent that runs local models on your PC — FellMentKE · 2026-11-19
- Apple opens iWork to external LLMs: Pages, Numbers, Keynote gain MCP support — dsp_ · 2026-11-19
- Hermes adapter gets GGUF release: matches xhigh thinking at a quarter of the tokens — yb2698 · 2026-11-19
- Anthropic engineer shares free 30-minute Claude Code SDK + GitHub Actions automation walkthrough — aftahi_ai · 2026-11-19
- Cost per completed task beats per-token pricing: LG U+ claims 30% agent savings — Dedap85 · 2026-11-19