146,010 requests in 13 days: OpenRouter endpoints flap wildly, 44 of 241 shift ≥20 points
ringarc · reddit · 2026-09-04
A Reddit user ran twice-daily sweeps of 241 OpenRouter endpoints (17 models, 52 providers) from Aug 20, sending 146,010 requests over 13 days and logging price, latency, and real completion rates.
Key findings:
- 109 availability shifts (error rate swinging ≥20 percentage points between consecutive days); 44 of 241 endpoints shifted at least once.
- 16 shifts ended at 100% error rate—11 of those had been at a clean zero the previous day.
- Crashes are rarely permanent: endpoints flap up and down repeatedly (e.g., one endpoint toggled between 0% and 100% error three times in four days).
The most troubling case is partial failure (53–63% error): a dead endpoint is obvious, but partial failures get swallowed by retry logic—latency climbs, costs rise, and nothing ever triggers a down status.
Caveats: single account, region, and request format, so 100% failure could mean rate limiting or deprioritization rather than outage; the error metric bundles rate limits, server errors, and timeouts. Providers are anonymized pending verification. Takeaway: a single test can't tell you an endpoint's real state, and pinning one provider is unreliable.
More from Infra
- Nvidia A100 pre-training pipeline goes live on a 24/7 public livestream — wavefnx · 2026-09-04
- ADSP Episode 302: Mark Saroufim on PyTorch, GPU MODE, and automating AI research — blelbach · 2026-09-04
- Spotify's Portal Cut Claude Code Token Usage by 90% With a Two-Mode Router — rseroter · 2026-09-04
- Vyact: Open-Source Desktop Workspace Unifying Local LLMs, RAG, and Browser Context — vyact · 2026-09-04
- At what context depth does KV quantization start to hurt? An F16 vs Q8/Q4 parity PoC — Slight_Analysis_5414 · 2026-09-04
- KV caching: the fundamental optimization behind autoregressive LLM inference — alec_helbling · 2026-09-04