"Just retry it" hides three LLM failure types — only one is fixed by retrying
ClickOk5811 · reddit · 2026-11-10
A Reddit post argues that wrapping LLM calls in a bare retry(3) conflates three failure classes needing different handlers:
- Transport/provider errors: timeouts, 429s, 5xx — the only bucket where retrying is clearly right, using exponential backoff with jitter and honoring Retry-After.
- Malformed output: invalid JSON, missing fields. Blind retries often repeat the error; validate against a schema and re-ask with the validation error included, with a hard cap. Constrained/structured output can shrink this bucket.
- Well-formed but wrong: hallucinated values, dropped clauses, confident misclassifications. Retrying can't help since nothing signals failure — you need rule checks, lookups against a source of truth, an independent second check, or human review for risky cases.
The author shares a three-stage code sketch and caveats: retries cost tokens and latency so caps and per-bucket failure logging matter; retrying non-idempotent actions after timeouts can duplicate side effects; bucket 3 has no generic solution.
More from coding & agent
- Laurence Moroney's 'Fine-Tuning AI' hits #1 new release in Machine Theory on Amazon — lmoroney · 2026-11-10
- Thom Wolf on software's future: Everflux, translucent sources, and bespoke AI-built features — Thom_Wolf · 2026-11-10
- PSAISuite: A PowerShell Toolkit for Calling Multiple AI Models via One Interface — dfinke · 2026-11-10
- Eric S. Raymond shares his AI coding workflow: Threadripper rig, Codex, no autonomous commits — esrtweet · 2026-11-10
- Claude Code settings files expose thousands of secrets on GitHub — Ars Technica AI · 2026-11-10
- Chollet backs 'AI CEO' Steve that runs acquired SaaS firms with P&L as objective function — fchollet · 2026-11-10