AI circuit breakers: stop agent infinite loops and budget blowouts at the reverse proxy layer

大模型之路 · wechat · 2026-10-19

The costliest autonomous-agent failure isn't a wrong answer—it's a tool-call loop burning tokens overnight, where each request looks legitimate and delayed billing means you notice too late. This piece walks through using an OpenAI-compatible reverse proxy as a circuit breaker.

Loop detection via sliding-window request fingerprinting returns 400 with infiniteloopkilled; per-account hourly/daily dollar budgets return 429 with budgetlimitbreached. Machine-readable codes let orchestrators route recovery differently instead of blind-retrying.
A microservice-style closed/open/half-open state machine distinguishes 'upstream down' from 'agent gone rogue' within seconds.
Division of labor: sandboxes govern execution, prompt guards govern output, MCP governs tool output size; the breaker governs volume and spend. Framework retries (LangGraph/CrewAI) handle single-point failures—the breaker handles systemic runaway, like a market trading halt.
Deployment tips: change one baseURL to adopt; choose gateway vs sidecar; start fingerprints at model+tool-name; run observation mode for a week before setting thresholds above normal variance.

Caveats: it governs quantity, not quality; false kills mean tuning fingerprints/thresholds; the proxy is a single point needing HA deployment; error-code semantics must stay stable or every orchestrator's recovery branch breaks.

Original post →

AI circuit breakers: stop agent infinite loops and budget blowouts at the reverse proxy layer

More from coding & agent