Stopping runaway agent spend before the bill: pre-call guardrails and three gaps found the hard way

MaverikSh · reddit · 2026-11-05

The author (who builds the Cognocient AI LLM gateway) shares engineering practices for agent cost guardrails: most cost tools report spend after the provider billed you, while a runaway loop can burn hundreds of dollars in minutes — so checks must run before a call goes out.

Core techniques:

Reserve before calling: atomically reserve each call's worst-case cost (from maxtokens), settle with real cost afterwards — otherwise 20 parallel calls all pass a budget check only one should pass
Per-run budgets: tag all calls with one run ID and enforce a dollar budget; step limits fail since 25 steps can cost $0.50 or $50
Dollar velocity, not just request rate: loops with ever-longer calls need a $/min limit
Behavioural loop detection: stop after N identical tool calls, same call/same result streaks, or error streaks
Read/write-aware degrading: hard-stop write-like tool calls when budget runs out, let reads finish on a cheaper model
A kill switch blocking everything

Three gaps in v1: clock-minute counting let spend straddle two minutes and pass a $30/min limit (fixed with 1-second buckets over a sliding 60s window plus a 10s burst window); the breaker didn't latch, so a blocked loop just waited and resumed (now refuses all calls for 5 minutes with Retry-After); a key-wide trip stopped every agent instead of the looping one (added optional per-run $/min limits).

Trade-off: it all sits in the request path (one base URL change, some latency); in-flight calls still complete and get billed. If you only need after-the-fact reports, a read-only billing integration is simpler.

Original post →

Stopping runaway agent spend before the bill: pre-call guardrails and three gaps found the hard way

More from coding & agent