Berth parks idle AI engines in RAM, cutting model swaps from 13.7s to 5.6s
Distinct-Pie2389 · reddit · 2026-10-24
Open-source Berth introduces a "park protocol": engines that can offload themselves (ComfyUI, vLLM, SGLang) move weights to system memory when idle and resume from RAM instead of disk.
- On an RTX 4090 with 64 GiB RAM, a 21GB chat model swap takes 5.6s with page cache (vs 13.7s cold from disk)
- A parked ComfyUI resumes in under 1s
- Switching to a 27B model takes 13.6–16.3s
Berth only parks when measured RAM covers weights plus a floor, and kills parked engines first under memory pressure.
More from Infra
- Redditor pushes back on local AI FUD: real inference speeds are 5x what the video shows — wojtek15 · 2026-10-24
- NVIDIA ships BioNeMo Inference Runtime v0.2 with RFD4-Proteina support — AllThingsApx · 2026-10-24
- ISTA-DASLab ships GSQ-RCO quantized DeepSeek-V4.1-Flash GGUFs, llama.cpp support still pending — Gold_Coconut9777 · 2026-10-24
- Sauers_ Mocks OpenAI Over Unstudied Multi-Agent Hardware — Sauers_ · 2026-10-24
- Local quant benchmark: new releases trail months-old DeepSeek/Qwen models — EmilPi · 2026-10-24
- OpenAI red team finds unauthenticated RCE in widely deployed Brotli compression library — gdb · 2026-10-24