Zhipu's InfraAgent boosted inference throughput 3.2x on 100k domestic chips in two weeks
APPSO · wechat · 2026-09-17
Zhipu chief scientist Tang Jie shared how GLM-5.3-driven InfraAgent helped deploy and optimize GLM-5.3-Flash on a cluster of over 100,000 domestic AI accelerators in just two weeks, raising end-to-end throughput 3.2x with per-token costs approaching mainstream NVIDIA GPU platforms.
The model itself performed much of the optimization work via a "dense feedback" mechanism: it located a TF32 precision-accumulation bug in KDA context-parallel paths (fix merged as FlashLinearAttention PR#1180), found that unreleased Python GIL in DeepEP caused over 30% KVTransfer overhead (reduced to under 1%), and lifted a redundant Decode Kernel by 1.71x by learning optimization patterns from SGLang and other open-source projects.
Zhipu frames this as an early form of recursive self-improvement: the model optimizes the system that serves the model, with engineers shifting toward designing feedback systems. The model previously ran anonymously as Ox-Alpha on OpenCode and OpenRouter, processing over 62 trillion tokens in six days.
More from AGI Musings
- "Causal patterns are all you need": MLStreet Talk debates whether the hard problem exists — burny_tech · 2026-11-02
- Dev says blog traffic fell from 700k to 7k reads as AI-generated job apps flood market — ericelliott_ · 2026-11-02
- Master foundational math to stay irreplaceable, argues mathematician — TivadarDanka · 2026-11-02
- Narayanan: labs racing toward Recursive Self-Improvement may hit Recursive Self-Extinction of their business model — random_walker · 2026-11-02
- Editor fully resigns from JMLR, arguing AI reviews beat rushed humans and journals need a rethink — roydanroy · 2026-11-02
- Alpha School investigation: 17 campuses on probation, Bill Ackman as shadow lead, Holmes involved — benjaminjriley · 2026-11-02