Why LLM spend limits fail during runaway loops, and the reserve-call-settle pattern that actually stops them

MaverikSh · reddit · 2026-10-24

The author breaks down why surprise LLM bills happen: provider limits are coarse (org/project, monthly), usage dashboards lag minutes to hours, maxtokens caps a single call not a loop, and code review misses runtime causes like leaked keys, ever-growing context, retry storms during outages, and input-triggered tool-call loops.

The pattern: reserve → call → settle. Before each request, atomically reserve the worst case (input tokens + maxtokens), settle with actual usage on success, release on failure. Pseudocode for a guardedcall is included.

Key details:

Reservations must be atomic (Redis Lua script or conditional SQL UPDATE) or concurrent workers all pass the check;
Reserve worst case, not average;
Charge the estimate for streams cut off mid-flight — you often never get final usage;
Retries must re-enter the budget check, with jittered backoff;
Use layered limits: per-key daily caps catch leaked keys, per-run caps catch loops, monthly team caps catch drift;
Alarm on spend rate vs. per-key baseline, not just totals;
Decide in advance what happens when the budget store is down (common compromise: a small per-key allowance).

Cheap wins: hard-cap agent iterations and tool calls, summarize history to stop context growth, split API keys per environment/service, alert on calls per minute.

Original post →

Why LLM spend limits fail during runaway loops, and the reserve-call-settle pattern that actually stops them

More from coding & agent