The Idempotency Trap in Agent-Driven Production Charges
anp2_protocol · reddit · 2026-07-18
An engineer deployed an agent to handle customer charges in production. However, a worker crashed after the charge succeeded but before the database write completed. Upon restart, it executed the same order again, resulting in the customer being charged twice.
The core issue identified: Idempotency keys and retry wrappers do not automatically solve consistency problems. If the idempotency key is only generated and stored in a local record that fails to save, the key is lost when the process crashes, leaving the provider unable to deduplicate.
The author suggests a more robust sequence:
- First, persist an "intent record";
- Call the payment system using a self-managed correlation ref;
- Finally, mark the status as complete.
Additionally, they emphasize that a post-restart state of "intent without complete" is an ambiguous state. The local database alone cannot determine whether the execution never happened or happened but failed to log. Therefore, you must query the external payment system, treating it as the "source of truth." If the provider doesn't support querying by correlation reference, manual reconciliation is the only option.
More from coding & agent
- GPT-6 Astra beats Factorio with enemies in 44 in-game hours at ~$4,500 API cost — liminal_bardo · 2026-09-11
- 105 hidden bugs, 2 repos: DeepSeek V4.1 Flash fixes 24 at $1.80 vs Opus 5's 27 at $51.33 — ChartsJournalX · 2026-09-11
- Investment Analyst Asks How to Build a Claude-Based Diligence Agent Stack — Careless_Tie2286 · 2026-09-11
- Treating agents like 50 First Dates: a 3-layer context system so every conversation doesn't start from zero — evielync · 2026-09-11
- Running the Firefox MCP on Android via Termux, ngrok, and mcp-proxy — Nervous-Strain7544 · 2026-09-11
- SmolVM open-sources persistent computer infrastructure for agents that outlive chat sessions — aniketmaurya · 2026-09-11