Open-source Berth auto-swaps AI engines on a single 24GB GPU with lease-based VRAM handoff
Distinct-Pie2389 · reddit · 2026-10-20
A Reddit user built Berth (GitHub), an open-source system service that solves the 24GB VRAM squeeze by automatically arbitrating GPU access: requests are queued by priority, the current holder is drained at a safe boundary, and the next engine only starts once the driver reports memory free.
- Author runs llama-swap (NInfer-4090), strata, ComfyUI and OpenWebUI on one RTX 4090, fixing daily pains like orphan node processes and ComfyUI squatting 500MB
- The daemon enforces one holder at a time, preventing collisions with games or stray workflows
- Berth also exposes engines behind a unified API endpoint, with full logging of who used the card and when
More from Infra
- PyTorch Conference poster to break down KV cache offloading: when CPU beats shared storage — PyTorch · 2026-10-20
- What Gen Models Run Well on a 3060/4060? Reddit Weighs Local Setups — Haghiri75 · 2026-10-20
- CoreWeave insiders from CEO to managers are dumping shares en masse, filing shows — SumitGup · 2026-10-20
- AMD open-sources TokenSpeed-Kernel: portable LLM inference kernels across GPUs — zhyncs42 · 2026-10-20
- Paper shows glass-embedded silicon trench capacitors enabling 50µm-thick vertical power delivery — jwt0625 · 2026-10-20
- ByteDance's DScout boosts diffusion-drafted speculative decoding with 3.84x-4.14x speedups — ByteDance · 2026-10-20