Provisioning a local inference server to a team: Tailscale + LiteLLM + per-user containers
Anarchaotic · reddit · 2026-11-10
A developer who built a Radiance inference server for their company asks how others provision team access. Their plan: Tailscale as the sole HTTPS entry point, LiteLLM for per-user keys with rate limits and usage logs, and Hermes as the harness with one container per user but a shared central skills repository. The open question is getting the harness to work across the team without the headache of manually procuring API tokens from providers like Google.
More from coding & agent
- OpenAI Agents API ships 12-hour prompt cache, async functions, and per-session budget control — OpenAIDevs · 2026-11-10
- pebl: an open-source minimalist terminal built for Claude Code sessions — damnGruz · 2026-11-10
- Garry Tan hails frontier model release days: Fable 5.2 gives every project a step-function boost — garrytan · 2026-11-10
- Meta's coding model Muse Ember goes live on OpenRouter for agent sub-task fan-out — alexandr_wang · 2026-11-10
- Meta's Muse Ember coding model is now available in OpenCode — alexandr_wang · 2026-11-10
- OpenRSI scales recursive self-improvement to SGLang-Omni, +7.6%–344% throughput via 14 merged PRs — ChengleiSi · 2026-11-10