Managing 5 LLM providers: which gateway handles billing, spend caps, and fallbacks?
InsuranceDependent71 · reddit · 2026-09-27
After growing from OpenAI-only to Anthropic, Gemini, DeepSeek, and Qwen plus an image model, the team juggles five dashboards, invoices, and key sets. Core requirements: one OpenAI-compatible endpoint, per-key spend limits to stop runaway agents, automatic fallback on 529s, and a single invoice.
Options evaluated: LiteLLM (self-hosted, solid, but needs an owner), Portkey, OpenRouter, and llmapi.ai (pay-as-you-go at provider list prices). The author is most curious how these gateways' fallbacks behave in production.
More from coding & agent
- Researcher Uses AI and Formal Verification to Unearth Three NetHack Bugs — davidbau · 2026-09-27
- Garry Tan touts AsideAI browser + MCP: let agents browse with your real credentials — garrytan · 2026-09-27
- Jev module adds a semantic if to PowerShell: model judges, script decides — dfinke · 2026-09-27
- Dev's token usage jumps 100x to 5B/day as multi-agent swarms go mainstream — gleech · 2026-09-27
- AI NPCs in GTA RP remember players: 20k conversations over 40 days on Polish server — JaynitMakwana · 2026-09-27
- AI Doesn't Kill Craft, It Moves It: Iterating With Agents Beats One-Shot Outputs — CSProfKGD · 2026-09-27