Cloudflare discloses GLM-5.3 cross-tenant KV cache leak: 155 responses affected
Cloudflare Blog · rss · 2026-10-27
Overview
On Sept 8, researcher Gadi Evron (Knostic) reported via HackerOne that Cloudflare Workers AI's @cf/zai-org/glm-5.3 leaked other users' session data in responses to very long prompts. The exposure window ran Aug 27–Sept 9; 155 responses were affected, all from prompts longer than 681,280 tokens. Cloudflare reproduced the issue, took the model offline within 40 minutes, and restored it the next day. No evidence of deliberate exploitation; no other model was affected.
Root cause: two overlapping issues
- Misconfiguration: the context window was set to 1,048,576 tokens, but the prefill KV cache pool only held 681,280. The mismatch was known pre-launch, but testing only validated HTTP codes and throughput, not output coherence.
- SGLang bug: in disaggregated prefill/decode mode, an over-length prompt was rejected and replaced with a one-token placeholder, yet prefill still generated and transferred the placeholder's KV cache. The decode node assumed the cache was fully overwritten, generated over stale pages, and leaked earlier requests' data.
Fixes and hardening
- Pool enlarged to 2.04M tokens (2× context), so valid requests never hit the rejection path.
- Engine patched so requests with pending rejections are retired at the disaggregation admission queue before any KV transfer; decode now verifies transferred token counts match reservations. Fixes upstreamed (#38704, #38935) and shipped in SGLang v0.5.21, with coordinated disclosure.
- Process change: KV pool size vs. context length is now checked from the running engine before deployment, not inferred from response codes.
More from Infra
- OpenAI's initial Vera CPU results: 51% more Python throughput vs leading x86 — firstadopter · 2026-10-27
- Berth: Open-Source Tool Lets Multiple AI Engines Share One GPU Without Fighting — Distinct-Pie2389 · 2026-10-27
- Cloudflare discloses cross-request KV cache leak in GLM-5.3 on Workers AI: 155 responses affected — michellechen · 2026-10-27
- Cloudflare discloses cross-request KV cache exposure in GLM-5.3 on Workers AI (155 responses) — wondersky · 2026-10-27
- One month fully local on a DGX Spark: daily tasks fine, code still needs Claude to check — michaelthatsit · 2026-10-27
- Cloudflare Workers adds spanContext, setStatus and updateName tracing APIs — irvinebroque · 2026-10-27