vLLM v0.31.0 uses CRIU snapshots to restore live TP1 engines without reloading weights
vllm_project · x · 2026-10-07
vLLM shipped v0.31.0, turning cold starts into process restoration. vllm snapshot create/restore uses CRIU to checkpoint and restore a live TP1 engine without reloading weights, while vllm preload is a daemon that keeps post-quantized weights resident across restarts. As the authors put it, cold start is now a Linux problem — a real win for inference services under frequent restarts.
Related event: vLLM v0.31.0 Ships CRIU Snapshots and Preload to Kill Cold Starts(3 posts)→
More from Infra
- Finland orders Google to halt work on two data centers over environmental reviews — KyeGomezB · 2026-10-07
- Z-Image CyberRealistic v9 GGUF Quant Fits 8GB VRAM, With x_pad_token Fix — Kordeyl · 2026-10-07
- EV charging firm Xeal plans 100,000 Nvidia GPUs at US sites for edge inference network — Nandu_alias_Parthu · 2026-10-07
- Toasty, the async Rust ORM by Tokio's creator, now fully supports Turso — glcst · 2026-10-07
- Kubernetes creators' Stacklok rebuilds the coding agent harness for the cloud with open-source Mecatl — Latent Space · 2026-10-07
- Ollama v0.40.0 auto-runs supported models on Apple's MLX runtime — lmoroney · 2026-10-07