GLM-5.3-powered Infra Agent took Chinese accelerators to full production traffic in 2 weeks with 3.2x throughput
victormustar · x · 2026-09-18
Jie Tang revealed that GLM-5.3-Flash went from first run on domestic Chinese accelerators to serving all production traffic in two weeks, with 3.2x end-to-end throughput — much of the work done by an Infra Agent powered by GLM-5.3, a model optimizing the system that serves it. Constraints included limited memory/interconnect bandwidth, 1M-token context, multimodal requests, and an immature stack with missing kernels. Key trades: compute for memory (ReplaySSM), communication for memory (intra-node tensor parallelism), precision for capacity (mixed INT8/FP8/BF16).
More from Infra
- The Absurd Reality of Finding GPUs: Q1'27 Ship Dates, Rugs and 50% Refunds — charles_irl · 2026-11-02
- Local AI computer maker Autonomous ships to 5 countries, prices up to $99,700 — dee_hw · 2026-11-02
- SGLang boosts multimodal inference >10% by replacing GPU memory bookkeeping with a dict lookup — charles_irl · 2026-11-02
- Big Tech's Future Data Center Leases Top $1.3T, 70% Added in Last Four Quarters — rohanpaul_ai · 2026-11-02
- Daniel Lemire: AI is exploding demand for boring compute, server CPUs to double — lemire · 2026-11-02
- Physical AI's real story: a 4-layer map of 15 companies behind humanoid robots — pdamodaran · 2026-11-02