Cloudflare details how it serves Kimi and GLM with KV-cache quantization and SGLang
michellechen · x · 2026-08-04
Cloudflare explains how it serves Moonshot’s Kimi K-series and Z.ai’s GLM efficiently on Workers AI close to users.
The post focuses on three production techniques:
- KV-cache quantization to cut memory use
- Weight compression to fit more requests on shared GPUs
- Cache protection to keep shared hardware fast under higher concurrency
Cloudflare says these optimizations let it support more customers at lower cost with no change in model accuracy, and that all experiments and production traffic run on SGLang, which it says is the best-performing serving framework it has tested.
More from coding & agent
- Hacking OpenAI Codex: Integrating DeepSeek with a Vision Model — yacineMTB · 2026-08-05
- Papers with Code Launches Frameworks: Discover AI Frameworks and Linked Papers — NielsRogge · 2026-08-05
- Stanford Course on Self-Improving AI Agents and New Survey Released — cong_ml · 2026-08-05
- OpenAI Codex Community Hackathon Announced in Bengaluru — tushaarmehtaa · 2026-08-05
- Essential Code Infrastructure Needed to Build Continuously at Full Speed with AI — StewartalsopIII · 2026-08-05
- Cloudflare OS: An Open-Source Agent Workspace with One-Click Deploy — irvinebroque · 2026-08-05