The Idempotency Trap in Agent-Driven Production Charges
anp2_protocol · reddit · 2026-07-18
An engineer deployed an agent to handle customer charges in production. However, a worker crashed after the charge succeeded but before the database write completed. Upon restart, it executed the same order again, resulting in the customer being charged twice.
The core issue identified: Idempotency keys and retry wrappers do not automatically solve consistency problems. If the idempotency key is only generated and stored in a local record that fails to save, the key is lost when the process crashes, leaving the provider unable to deduplicate.
The author suggests a more robust sequence:
- First, persist an "intent record";
- Call the payment system using a self-managed correlation ref;
- Finally, mark the status as complete.
Additionally, they emphasize that a post-restart state of "intent without complete" is an ambiguous state. The local database alone cannot determine whether the execution never happened or happened but failed to log. Therefore, you must query the external payment system, treating it as the "source of truth." If the provider doesn't support querying by correlation reference, manual reconciliation is the only option.
More from coding & agent
- Harness engineering is emerging as the execution layer for reliable AI agents — Pavan_Belagatti · 2026-07-21
- DevFest Lisbon keynote will cover Google AI Studio’s latest vibe coding and agentic AI features — gerardsans · 2026-07-21
- Daniel Hanchen’s 2-hour workshop covers open models, reward hacking and RL — danielhanchen · 2026-07-21
- A Codex joke turns into a recursive debate about Cloud Codex — Dimillian · 2026-07-21
- AI-assisted development is widening security backlogs, so teams should measure risk velocity — WeldPond · 2026-07-21
- Base44 offers $10,000 for AI agent builds on one-command backend — Base44_Sam · 2026-07-21