Cloudflare details how it serves Kimi and GLM with KV-cache quantization and SGLang
michellechen · x · 2026-08-04
Cloudflare explains how it serves Moonshot’s Kimi K-series and Z.ai’s GLM efficiently on Workers AI close to users.
The post focuses on three production techniques:
- KV-cache quantization to cut memory use
- Weight compression to fit more requests on shared GPUs
- Cache protection to keep shared hardware fast under higher concurrency
Cloudflare says these optimizations let it support more customers at lower cost with no change in model accuracy, and that all experiments and production traffic run on SGLang, which it says is the best-performing serving framework it has tested.
More from coding & agent
- Hands-on With Opus 5.5: Same Feel as 4.6, More Power, ~40% Cheaper Than Opus 5 — EricBuess · 2026-09-23
- Anthropic ships Claude Opus 5.5: cheaper by up to 40% and pulling Codex users back to Claude — danshipper · 2026-09-23
- Auto-research frameworks may be overkill: one paragraph prompt gets similar results — generativist · 2026-09-23
- Why Nautilo built its own mobile apps instead of piggybacking on chat apps — Dan_Jeffries1 · 2026-09-23
- LangChain shares walkthrough of Typesafe's new jev model and how to plug it into agents — LangChain · 2026-09-23
- One prompt turns Grok into a full marketing team: ads, videos and influencer outreach — JaynitMakwana · 2026-09-23