Zhipu says a GLM-5.3 agent optimized its own inference stack, 3.2x throughput on 100k domestic chips

SinclairWang1 · x · 2026-09-17

Zhipu revealed that all production inference for GLM-5.3-Flash now runs on over 100,000 domestic AI accelerators, with 3.2x end-to-end throughput gains achieved in under two weeks — and much of the optimization was done by a GLM-5.3-powered Infra Agent, a model optimizing the very system that serves it.

Hard constraints, constant trades

Domestic accelerators have limited memory/interconnect bandwidth and an immature software stack with missing kernels and guesswork documentation
The model must serve 1M-token context and multimodal requests
Every optimization was a trade: compute for memory (ReplaySSM), communication for memory (intra-node tensor parallelism), precision for capacity (mixed INT8/FP8/BF16 caching), architecture separation for scheduling freedom (encode-prefill-decode disagg)
Per-token cost on domestic chips now matches mainstream NVIDIA GPUs

The key method: dense feedback

CEO Jie Tang notes agents rarely get stuck because they can't write code — they get stuck because "throughput dropped 20%" doesn't say which layer broke, what to verify, or what to test next: a sparse-reward credit-assignment problem, with each end-to-end benchmark run taking hours. Zhipu made senior engineers' implicit "process rewards" explicit via layered verification interfaces: correctness (is it right), system behavior (where time goes), and performance (which approach wins under what conditions) — each signal local, cheap to obtain, and objectively verifiable.

With this, the agent found and fixed three real engineering bugs, including TF32 rounding-error drift accumulating through chained state-matrix merges in the KDA kernel's context-parallel path; the fix is merged into the open-source Flash Linear Attention project.

Original post →

Zhipu says a GLM-5.3 agent optimized its own inference stack, 3.2x throughput on 100k domestic chips

More from coding & agent