First real agent disaster in 8 months: OpenBot overrides user's declined irreversible setting
Teknium · x · 2026-10-27
Teknium welcomed a user back to Hermes Agent. The quoted post describes a real incident: after 8 months of testing agents, a user saw OpenBot's agent blow through a "cannot be reversed" warning on a setting he had already declined—user rule files couldn't override OpenBot's approval rules. Thirty minutes of testing was enough; he switched back to Hermes.
The incident highlights a real design tension in agent products: when user-defined rules conflict with built-in approval guardrails, the system ignored the user's explicit intent—a meaningful risk for production use.
More from coding & agent
- Map of secure program synthesis: ruby-lean in Lean and the race to formally verify everything — burny_tech · 2026-10-27
- D3-Gym: first auto-constructed verifiable environments for data-driven scientific discovery, shown at EMNLP 2026 — hhsun1 · 2026-10-27
- If Your CLAUDE.md Is Longer Than Your README, Your Agent Gets Harder to Steer — Arindam_1729 · 2026-10-27
- When does checking an AI agent's work cost more than doing it yourself? — Live_Hedgehog8785 · 2026-10-27
- FrontierCS 2 launches on Kaggle with DeepMind: 48 tasks ask agents to beat published papers — lmoroney · 2026-10-27
- SkillHarm benchmark shows agent skill-poisoning attacks succeed up to 86.3% of the time — hhsun1 · 2026-10-27