Skip big-provider AI on your phone: llama.cpp + OpenWebUI + Tailscale local setup
GodComplecs · reddit · 2026-10-02
A Redditor shared a simple setup for using local LLMs from your phone instead of big providers:
- Run llama.cpp backend (0.0.0.0:8080) + OpenWebUI (0.0.0.0:8081) on your PC, with local model search enabled
- Use Tailscale to reach your home-hosted models from the phone
- The "secret sauce" for 24GB VRAM: run unsloth's Qwen 3.6 in instruct/non-thinking mode with proper settings for a near-cloud experience
- The author says this fully replaces Google AI Mode etc.; terminal tools like opencode can also route through it (though he hasn't done much agentic stuff)
More from Infra
- tandem.cpp v0.3.0: 2-6x faster prefill for models too big for VRAM, Maya DeepSeek-V4.1 support — smflx · 2026-11-16
- Developers poll real costs of running coding agents on remote VMs for work — radim11 · 2026-11-16
- Datadog's new gateway returns 402 when a model's budget runs out—so agents just retry — YvesMulkers · 2026-11-16
- Max-quality local Minimax H3 video on a 4090: community-tested int8 setup — newaccount47 · 2026-11-16
- Local V2V outfit transfer on a 16GB M3 MacBook Air: 15.7 min cut to 2.4 min — might_thinker · 2026-11-16
- FlashStates hits 3.14x vLLM prefill throughput for hybrid-architecture agentic inference — 机器之心 · 2026-11-16