Agent claimed it saved a file without calling the tool: one dev's fix for action-hallucination
Stunning-Sherbet1853 · reddit · 2026-10-14
A developer building a local Windows agent hit a classic failure: a 4B model replied "Saved" with a file path, but the file-writing tool was never called and nothing existed at that path.
Root causes:
- The router didn't treat "give/pass/send ... as a file" as a file request, so it fell into a no-tool, text-only path;
- Nothing on that path verified whether a claimed action actually happened.
Fixes:
- Match the request shape (format name + as + give/pass/send) instead of adding verbs one by one; tested old vs. new rules over 1,931 real requests — the new rule caught 2 more correctly, with no false positives;
- Before a no-tool answer ships, if it contains "saved"/"created" plus a filename or path, check the file exists; if not, discard and re-route through the tool path.
The author notes the same say/do mismatch appeared twice before ("I checked" with zero tool calls; 1 of 9 direct answers was a copy of an earlier reply), and asks whether verification belongs at the output gate, whether one generic rule (block claimed actions without a matching tool record) beats per-type checks, and whether strict verification ever discards good answers.
More from coding & agent
- A Full QC Pipeline for RL Data: Intake Review, Active Testing, Packaging — joecole · 2026-10-14
- Google offers 10,000 scientists free 6-month top-tier Antigravity plans — danielrock · 2026-10-14
- CineRecon Benchmark: 12 Frontier Models Struggle to Reconstruct Editable 3D Scenes from Video — hkuhk · 2026-10-14
- Meituan's ClawLoop: Self-Improving Agents via Teacher-Free Harness-Native Training Loops — meituan-longcat · 2026-10-14
- Grok Bot's big leap: full breakdown of latest updates and autonomous business test — elonmusk · 2026-10-14
- Indie dev cuts homepage job list load from 1.8s to 0.1s in performance push — DanielLockyer · 2026-10-14