Open-source warmctl restores a warmed-up vLLM engine in ~37s on 8×H100
kbesplv · reddit · 2026-10-13
warmctl is a new open-source tool that snapshots a fully warmed-up vLLM engine and restores it in seconds on a different host with multi-GPU support — enabling true scale-to-zero on your own GPUs, unlike hosted offerings from Modal, Inferize, or RunPod FlashBoot. How it works: vLLM sleep mode offloads weights, CRIU checkpoints the process, CUDA checkpoint API saves GPU state, cuInterpose handles multi-GPU, and snapshots ship as OCI images. Demo: GLM-5.3-Flash served on 8×H100 in 37s.
More from Infra
- Broadcom optical roadmap: 1,000+ fibers per chip package, 400G direct drive — firstadopter · 2026-10-13
- Broadcom optical roadmap: TH6 hits 102.4T in 2026, next-gen OCI by 2028 — firstadopter · 2026-10-13
- Noise: a speedrun for pretraining transformers without backprop — madhavsinghal_ · 2026-10-13
- P100 x2 + Qwen3: Vibe-Coded llama.cpp Patches Nearly Double Throughput — Mrinohk · 2026-10-13
- Leaked OpenAI slides: ChatGPT tops 1B weekly users, compute grows 10X every two years — firstadopter · 2026-10-13
- Applied Materials unveils EPIC Center as analyst sees WFE demand soaring — BenBajarin · 2026-10-13