Ray + vLLM clusters expose unauthenticated control-plane ports, benchmark finds
No-Peanut-6988 · reddit · 2026-09-29
Benchmarking a multi-node Ray + vLLM inference cluster on AWS EKS under vendor defaults, a team found: 15 of 17 active listening sockets never appeared in declared containerPort manifests; a neighbor container in an unrelated namespace could reach Ray GCS (6379) and raylet RPCs (10002–10006) over unauthenticated cleartext gRPC; four default scanners (Trivy, Checkov, Kubescape, kube-linter) missed it entirely since they don't inspect the RayCluster custom resource; and a single-GPU vLLM deployment exposed 26 unauthenticated API routes.
Mitigations: an ingress default-deny NetworkPolicy blocked all neighbor-pod access to the Ray control plane with no measurable latency. WireGuard encryption (Cilium chained with AWS VPC CNI) carried the workload but cost 3.4%–6.5% throughput and 3.2%–8.4% latency under load. Full report and hardening guide available.
More from Infra
- Celesto: open-source persistent microVM computers for AI agents, boots in 500ms — aniketmaurya · 2026-09-29
- Redditor runs gpt-oss-120b across a phone, three Macs and two Windows PCs — ANR2ME · 2026-09-29
- BioNeMo team boosts Mixtral-8x7B training throughput 2.21x vs HF BF16 baseline — AllThingsApx · 2026-09-29
- DeepSeek's elastic compute team is hiring heavily, shares sandbox infra for large-scale agent training — teortaxesTex · 2026-09-29
- Developer slams third-party inference providers: Gemini up 10x, Luna 15s latency — julianharris · 2026-09-29
- Nereus: adaptive parallelism boosts 8B PPO throughput up to 7.27x over OpenRLHF — Songlin Jiang · 2026-09-29