FailureAtlas: Open-Sourcing an LLM Gateway Failure Catalog
its_vayishu · x · 2026-07-20
@metriqual open-sourced FailureAtlas, a failure taxonomy and case catalog for multi-vendor LLM gateways, focusing on hidden production environment issues like "HTTP 200 but the app is actually broken."
They categorized failures across 5 architectural layers, highlighting the Silent category as the most dangerous. Examples include async race conditions causing cross-tenant context leakage, parallel tool call index collisions in SSE streams, concurrency races in state management, and issues invisible to standard APM. The author stresses the need for semantic observability and invites the community to submit PRs to expand this open directory.
More from coding & agent
- GitHub review bot hits its PR limit and forces a 39-minute cooldown — DanielLockyer · 2026-07-22
- Max reasoning effort appears to be mobile-only in Codex Remote, not desktop — GabGarrett · 2026-07-22
- A Reddit demo argues online stores should expose carts and pricing through MCP — gelembjuk · 2026-07-22
- Open-source AI SDK provider routes Vercel apps through a local Codex subscription — lgrammel · 2026-07-22
- Codex vs Claude Code: Which Is More Popular? — jxnlco · 2026-07-22
- CodeRabbit uses layers, diagrams and a chat agent to rethink code review — _jaydeepkarale · 2026-07-22