AssemblyAI Adds Circuit Breaker to LLM Gateway: Skip Degraded Models Before Your Request Fails
AssemblyAI · x · 2026-11-20
AssemblyAI introduced a beta circuit breaker for its LLM Gateway. Before a request is sent, it checks the model's latency and failure rate; if the model is degraded beyond your thresholds, the gateway skips it and routes to your next fallback model.\n\nUnlike fallbacks, which react to requests that already failed, the circuit breaker reacts to a model that is degrading — so a slow or failing model costs nothing extra. It works on chat completions and decisions requests and is enabled via fallbackconfig.circuitbreakerconfig with percent or failure thresholds.
More from coding & agent
- Vercel CEO: outcome-oriented engineers are now 1000x, and coding is AI's #1 killer app — tlakomy · 2026-11-20
- LangChain Celebrates Fundraise With Mini Launch Week: Trace Governance, Tool Gateway, LangSmith for Coding Agents — hwchase17 · 2026-11-20
- Techtree Packages Skills, Evals and Training Envs to Provably Upgrade Business Agents — seanwbren · 2026-11-20
- Open-source F1 MCP Server: telemetry, tire wear and Monte Carlo strategy — modelcontextprotocol · 2026-11-20
- Maindex Smart Memory Service gives agents memory via four explicit tools — modelcontextprotocol · 2026-11-20
- Linear COO: $100M ARR, 177% NRR, and agents now create half its work — Latent Space · 2026-11-20