vLLM cold starts on RunPod cut to 14s: DMTCP snapshots where CRIU fails
kbesplv · reddit · 2026-11-01
warmctl shipped a lightweight second backend that snapshots a warmed-up vLLM process to a network volume, letting new pods restore in seconds instead of minutes. Demo: Qwen3-32B on 2×L40S serving in 14s (TP=2) via DMTCP, since CRIU doesn't work on RunPod. Practical for self-hosting LLMs on cloud GPU platforms.
More from Infra
- Is llama.cpp the new Ollama? Reddit debates the fragmented local LLM engine scene — fintip · 2026-11-01
- Researcher teases talk synthesizing a decade of large-scale training systems — _arohan_ · 2026-11-01
- Meme: writing an LLVM backend for a fully undocumented GPU architecture to ditch closed-source nvcc — mike64_t · 2026-11-01
- Enterprises are distilling domain models, cutting costs from dollars to cents with no accuracy loss — himanshustwts · 2026-11-01
- Prediction: Local Models Will Be the Biggest AI Story of 2027, Disrupting Inference Business — ZeYanjie · 2026-11-01
- Reddit bench: EPYC + multi-3090 rig runs Qwen Q8 at 94 tok/s with near-linear scaling to 8 streams — sleight42 · 2026-11-01