NVIDIA ships AI Cluster Runtime: validated Kubernetes recipes for its Dynamo inference stack
nvidia · x · 2026-11-03
NVIDIA introduced AI Cluster Runtime (AICR), packaging its Dynamo distributed inference framework plus supporting infrastructure into validated, reproducible recipes, with a full Kubernetes walkthrough.
- Pins GPU driver stacks, Dynamic Resource Allocation, gang scheduling, node health signals, and Prometheus metrics
- Dynamo 1.4.2 uses TCP for the request plane and ZMQ for KV-cache events; both the native Frontend router and the Gateway API Inference Extension (GAIE)/EPP path support cache-aware routing
- Five-step flow: snapshot cluster → generate inference recipe → render deployment bundle → deploy via Helmfile/GitOps → validate before app workloads
- Team presenting at KubeCon North America
More from Infra
- Gap runs loss-prevention AI entirely on Mac minis, cited on Apple's Q4 earnings call — akumar · 2026-11-03
- A single GPU now out-earns the US federal minimum wage of $7.25/hr — blelbach · 2026-11-03
- Inference engineering called 'the most lucrative, highest-leverage specialty skill on earth' — AccBalanced · 2026-11-03
- Apple says memory price hikes are 'unprecedented,' purchase obligations jump $33B in one quarter — firstadopter · 2026-11-03
- Peking University's MAServeBench benchmarks LLM serving with multi-agent workloads — PekingUniversity · 2026-11-03
- ProVe flips speculative decoding: large proposer, small verifier yields 2.5-2.7x speedup — Ziqi Jin · 2026-11-03