PROCS paper: parallel snapshot compression cuts inference cold-start latency up to 44.7%
charles_irl · x · 2026-11-04
A paper from Doubleword, Oxford, NVIDIA and collaborators introduces PROCS, a parallel restore-optimized compression method for CRIU snapshots targeting inference cold-start costs. Evaluated on five SGLang models with CRIU-GPU: 256 KiB blocks shrink snapshots 12.5–45.5% and restore-to-first-token latency 7.2–44.7%; 4 MiB blocks cut size 62% and checkpoint time 12.4%; 16-thread decompression on Qwen3.5 27B FP8 cuts restore latency another 25.8%. Practitioners report using such research to start generating tokens as soon as autoscaled GPUs come up.
More from Infra
- OpenAI launches grid-protection program to ease data-center strain on the power network — emmanuelvivier · 2026-11-04
- 8 free GitHub repos with 792K stars that take your AI off Big Tech's servers — alex_verem · 2026-11-04
- OpenClaw cut its ~$1M/month CI bill by over 95% with adaptive agentic testing — vincent_koc · 2026-11-04
- LlamaIndex stacks 17 small optimizations for ~10x faster OCRia-1 token generation on a single B200 — llama_index · 2026-11-04
- phonon: WebGPU browser TTS hits 26x real-time on iPhone with a 44MB model, open sourced — blixt · 2026-11-04
- GlobalWafers at full capacity across all wafer sizes, ASP hikes expected from 2027 — BenBajarin · 2026-11-04