ngrok AI Gateway routes local LLMs and frontier models through one URL
anthara_ai · x · 2026-09-04
ngrok's AI Gateway bets on "never bet on one model": bring self-hosted models and every frontier provider into a managed gateway with built-in observability and access control.
Dead-simple setup: change your SDK's baseURL to https://gateway.ngrok.ai and swap the API key — works with OpenAI, Anthropic, and Vercel AI SDKs. Requests can route to a self-hosted model first (e.g. a local Qwen GGUF) and fall back to public providers like gpt-5.4 or claude-opus-4-6.
Key capabilities:
- Local LLMs: connect any locally reachable model via private connectivity — no public IPs or inbound ports needed;
- BYOK: drop in your existing OpenAI/Anthropic keys, keep your current rates, manage them in one place;
- Access control: separate access keys per app/developer, each scoped to allowed providers and models;
- Observability: tokens, latency, errors, and cost rolled up across every call by app, dev, and model — not just provider dashboards.
Related event: ngrok launches AI Gateway: one URL to route cloud and self-hosted models(7 posts)→
More from coding & agent
- Screenshot table to Excel in PowerShell: LLM only locates, code does the rest — dfinke · 2026-10-21
- One line of PowerShell turns any screenshot table into real Excel data in 4 seconds — dfinke · 2026-10-21
- Screenshot to spreadsheet in 4 seconds with one line of PowerShell — dfinke · 2026-10-21
- A mitm proxy that rewrites robots.txt to Allow: all gets you really far — generativist · 2026-10-21
- Anaconda ships MCP to stop coding agents from hallucinating malicious dependency packages — anacondainc · 2026-10-21
- 5-person team uses Claude and 300+ agents to design 318,484 proteins in 64 hours — 141_1337 · 2026-10-21