ngrok launches AI Gateway: route local LLMs and OpenAI through one URL
anthara_ai · x · 2026-09-03
- A demo shows connecting a local self-hosted model and OpenAI/Anthropic through a single managed gateway: no deployment, no VPN, no separate integrations—just point your SDK's baseURL at gateway.ngrok.ai and swap the API key.
- Switching models means changing one model name, with fallback chains supported (try a local GGUF model first, fall back to gpt/claude).
- Key features: private connectivity to self-hosted LLMs without public IPs, BYOK for existing provider keys, scoped per-app/per-developer access control, and unified observability of tokens, latency, and cost.
- The pitch is "never bet on one model": multi-model routing as a managed service instead of DIY infrastructure.
Related event: ngrok launches AI Gateway: one URL to route cloud and self-hosted models(7 posts)→
More from coding & agent
- Screenshot table to Excel in PowerShell: LLM only locates, code does the rest — dfinke · 2026-10-21
- One line of PowerShell turns any screenshot table into real Excel data in 4 seconds — dfinke · 2026-10-21
- Screenshot to spreadsheet in 4 seconds with one line of PowerShell — dfinke · 2026-10-21
- A mitm proxy that rewrites robots.txt to Allow: all gets you really far — generativist · 2026-10-21
- Anaconda ships MCP to stop coding agents from hallucinating malicious dependency packages — anacondainc · 2026-10-21
- 5-person team uses Claude and 300+ agents to design 318,484 proteins in 64 hours — 141_1337 · 2026-10-21