Berth: Open-Source Tool Lets Multiple AI Engines Share One GPU Without Fighting

Distinct-Pie2389 · reddit · 2026-10-27

Open-source project Berth (223 stars) manages a single GPU at the driver level so multiple AI inference engines can time-share one card:

Queuing & priorities: llama.cpp, Ollama, vLLM, SGLang, ComfyUI and LM Studio take turns by priority (foreground/interactive/batch); the next engine only loads after the driver shows memory free, so two models never overlap.
One OpenAI-compatible port: name a model and Berth starts its engine, queues and proxies the request. ComfyUI hands over the card between jobs, never mid-render; RAM-offloadable engines park in memory (ComfyUI parks in under 1s).
Coexists with games & creative apps: automatically yields the card while gaming (Steam, Epic, GOG, etc.); Blender, Resolve, OBS and Premiere memory is labeled, not flagged.
Model library & observability: pull models from Hugging Face/CivitAI with resume and hash checks, local web dashboard, Prometheus metrics, plus an agent skill teaching Claude Code/Codex to ask Berth for the card instead of fighting over it.

Linux + NVIDIA is fully supported; Windows 10/11 NVIDIA is experimental with full CI test suite; AMD Linux experimental. Measured on an RTX 4090: a 21GB model loads in 5.6s; ComfyUI→27B switch takes 13.6–16.3s; on a five-GPU rig a voice assistant borrowing 6.6GiB from a streaming MoE server saw max 0.38s pause between tokens.

Original post →

Berth: Open-Source Tool Lets Multiple AI Engines Share One GPU Without Fighting

More from coding & agent