Agents built a custom inference engine: AMD MI350X serves GLM 5.3 FP8 37% faster than NVIDIA B200
yisongyue · x · 2026-10-28
Asari AI Labs reports its agents built a custom inference engine in just a few days that serves GLM 5.3 FP8 on AMD MI350X, 37% faster than on NVIDIA B200/B300 8-GPU nodes.
On SemiAnalysis' AgentX benchmark it achieves a p90 interactivity of 352 tokens/sec/user — even matching the best result for lower-precision FP4 models — with all outputs consistent with the stock implementation. Charts show MI350X + Asari AI Engine at 351.7 tok/s/user, well above B200·SGLang FP8 (255.8), B300 (251.9), and even GB300·Dynamo SGLang FP4 (342.5). Reposter Yisong Yue highlights big potential in this agent-built-inference-engine approach.
More from coding & agent
- N-queens solved entirely in Python type annotations with ty, each solution a tuple — charliermarsh · 2026-10-28
- MCP gap: services can't push to agents, dev argues spec needs server-initiated events — braelyn_ai · 2026-10-28
- Adding Push to MCP: A New Attack Surface and a Design Sketch — braelyn_ai · 2026-10-28
- Swarms Cloud revamps Models page aggregating 2,000+ LLMs with pricing — KyeGomezB · 2026-10-28
- A new open source model: closed to pull requests, open to remix — kieranklaassen · 2026-10-28
- CC Switch tutorial: rotating multiple Claude official subscriptions and ban-risk notes — dotey · 2026-10-28