Sarvam AI serves GLM 5.3 with MXFP8 on vLLM, halving TTFT and doubling throughput

HarveenChadha · x · 2026-11-16

Indian AI company Sarvam AI announced it serves GLM 5.3 with MXFP8 precision on vLLM, halving time-to-first-token and doubling throughput.

Harveen Chadha adds why faster inference matters beyond customers: for evals, faster inference means faster feedback loops; for post-training, faster rollouts in RL translate into more experimentation time and higher research velocity. He says he contributed to token factory work and is applying those learnings to RL.

Original post →

More from Infra

Infra channel →