Vertex AI users report Google secretly breaking KV cache, driving up token bills
niwatoriking · reddit · 2026-11-05
A r/googlecloud thread warns Vertex AI customers to audit their bills: the author claims Google has been silently breaking KV cache hits, inflating token costs, and explains how to request a refund. A practical warning for teams running LLM inference on Vertex AI to check cache hit rates and billing (details per the original thread, pending Google's response).
More from Infra
- Malaysia Approves 117 Data Centers, Power Demand to Hit 9GW in Five Years — basedjensen · 2026-11-06
- TWiAI Podcast: Altman's DRAM Complaint and Bittensor Subnets — markjeffrey · 2026-11-06
- Splitting one 27B across two 3090s was slow: one model per GPU hits 243 tok/s combined — StewartalsopIII · 2026-11-06
- Shawn Hymel's new tutorial: using Meshtastic devices as IoT sensor nodes with Python — ShawnHymel · 2026-11-06
- UBS: Aspeed Has Secured 2027 BMC Orders on Strong Agentic AI Server Demand — BenBajarin · 2026-11-06
- Firecracker VM escape disclosed: researchers claim root execution on the host — moyix · 2026-11-06