Your LLM fallback never fires for providers that stall before the first token

FallElectronic3774 · reddit · 2026-11-11

A Reddit post highlights a common blind spot in LLM fallback logic: retries fire on exceptions, handling 429s and 500s, but not providers that accept the request and stall before the first token — nothing throws, so no fallback happens. The fix is a separate time-to-first-byte timeout that must fire before anything is written to the client; after the first chunk is out, a retry just yields a second answer next to a half-finished one.

Original post →

Your LLM fallback never fires for providers that stall before the first token

More from coding & agent