Dev Builds 'Local ChatGPT in a Box' on One RTX 5090, Sharing Every Workaround Along the Way
valdev · reddit · 2026-09-18
Reddit user valdev shares LumaBrowser, a self-built "local Anthropic/ChatGPT in a box" running on a single RTX 5090, with one-click toggling between LLM, image, editor, and video models. Key engineering details:
- RAM-pinning model files (bypassing Windows's lack of native ramdisk) to cut load times to 10 seconds
- Forked llama.cpp to fix read-speed bottlenecks, then RAM-pinned context for hot reload without losing conversation
- Electron-based automation platform: network interception to scrape JSON report data from API-less sites, plus aggregating Slack/email/web notifications with an AI client that groups and pushes important messages every 10 minutes via ntfy
- An MCP server exposing it all, with LLM fallbacks for brittle selectors — natural-language element lookup that reports back the corrected selector
- A modular extension architecture where extensions can extend each other
A dense engineering retrospective with plenty of reproducible tricks for local model serving, browser automation, and agent workflows.
More from coding & agent
- Give your coding agent a good feedback loop — output quality improves 10x — DanielLockyer · 2026-09-18
- Mollick: Claude Projects Spins Up 18 Specialist Agents to Crack Historical Mysteries — emollick · 2026-09-18
- Building a Real-Time AI Chat, Vision and Voice iOS App with Gemini 3.8 Live — amos_gyamfi · 2026-09-18
- A Correct Answer Can Still Invalidate Your AI Agent Eval, Microsoft Dev Blog Warns — WirelessLife · 2026-09-18
- Google and Speakeasy open-source their entire OpenAPI SDK generation suite — _philschmid · 2026-09-18
- Agentic apps need different MLOps: caching, model routing and evals for low latency — kmeanskaran · 2026-09-18