Managing 5 LLM providers: which gateway handles billing, spend caps, and fallbacks?
InsuranceDependent71 · reddit · 2026-09-27
After growing from OpenAI-only to Anthropic, Gemini, DeepSeek, and Qwen plus an image model, the team juggles five dashboards, invoices, and key sets. Core requirements: one OpenAI-compatible endpoint, per-key spend limits to stop runaway agents, automatic fallback on 529s, and a single invoice.
Options evaluated: LiteLLM (self-hosted, solid, but needs an owner), Portkey, OpenRouter, and llmapi.ai (pay-as-you-go at provider list prices). The author is most curious how these gateways' fallbacks behave in production.
More from coding & agent
- Baseten's Guide: Evaluate Open Models for Coding Agents on Your Own PRs — baseten · 2026-11-16
- Google Research open-sources PaperOrchestra, a multi-agent framework for writing AI papers — google-research · 2026-11-16
- ffmpeg-skill gives coding agents local video editing via 42 FFmpeg tools and MCP — kajisho5 · 2026-11-16
- Agents Now Reply to Slack Messages on Their Own, Resolving a Prod Incident Ghost-to-Ghost — iamrobotbear · 2026-11-16
- Engineer's Interview Prep Method: Use Claude to Find Which Answers Undersell Your AI Work — brandon_galang · 2026-11-16
- DeepSeek open-sources its app frontend "chat-demo", already at 3.9k stars — NielsRogge · 2026-11-16