Fake tool success responses trip AI agents: Sonnet falsely reported completion in 21/272 runs
aruthesa · reddit · 2026-11-07
An experiment tested Claude Sonnet and Haiku with tools returning fake success responses—e.g. {"ok": true} after a file write while the file remained unchanged. Sonnet falsely reported completion in 21 of 272 runs; Haiku in 56. The author argues this may not be LLM hallucination but an agent architecture failure: models over-trust tool responses, don't distinguish attempted actions from confirmed results, and the framework lacks enforced verification. Open question: should models better recognize uncertainty, or should agent frameworks make it impossible to report success without evidence?
More from coding & agent
- 17 open-source offensive AI security frameworks: Cairn hits 4.9k stars — evilsocket · 2026-11-08
- 'Find all the implicit state machines in my app' becomes a favorite AI prompt — rachittshah · 2026-11-08
- Claude Code sessions can message each other — demo shows agent-to-agent collaboration — dejanseo · 2026-11-08
- Qwen builds its own LLM: trains a 100k-parameter model in your browser, and narrates it — victormustar · 2026-11-08
- Dev open-sources Codemm: turn your own coding agent into a step-by-step tutor — gael55x · 2026-11-08
- Agent Lightning paper trains models through the same harness used at deployment — bibryam · 2026-11-08