AssemblyAI Adds Circuit Breaker to LLM Gateway: Skip Degraded Models Before Your Request Fails

AssemblyAI · x · 2026-11-20

AssemblyAI introduced a beta circuit breaker for its LLM Gateway. Before a request is sent, it checks the model's latency and failure rate; if the model is degraded beyond your thresholds, the gateway skips it and routes to your next fallback model.\n\nUnlike fallbacks, which react to requests that already failed, the circuit breaker reacts to a model that is degrading — so a slow or failing model costs nothing extra. It works on chat completions and decisions requests and is enabled via fallbackconfig.circuitbreakerconfig with percent or failure thresholds.

Original post →

More from coding & agent

coding & agent channel →