Open-source TrnRT engine serves GLM-5.3-Flash on one Trainium2 box at 3.57M tokens/min
ycombinator · x · 2026-11-05
RunAnywhereAI says AWS Trainium had zero support for the latest open models, so they wrote their own megakernels and open-sourced TrnRT, a Trainium inference engine.
Headline result: running Z.ai's 320B GLM-5.3-Flash on a single Trainium2 box (TRN2.48XLARGE) delivers 3.57M tokens per minute, with decode step improved from 32.2ms to 24.2ms (6.1ms memory floor), measured on real coding-agent traffic via SemiAnalysis's AgentX.
Context: Anthropic is betting up to 5 GW on AWS Trainium, and AWS's Andy Jassy announced TorchNeuron now runs existing PyTorch code natively on Trainium with one config change — the open-source community is filling in the inference-software gap for this chip ecosystem.
Related event: Open-source TrnRT engine runs 320B GLM model on a single Trainium2(2 posts)→
More from coding & agent
- Dev Open-Sources optimAIzr: 56% of AI Coding Costs Come From Rereading Context — stichstichstich · 2026-11-05
- Spring creator Rod Johnson launches Embabel Worlds, an AI platform that auto-builds knowledge graphs — springrod · 2026-11-05
- Cognition Launches SWE-2 Flash: 3x Faster at One-Fifth the Price — winer666 · 2026-11-05
- lm-eval Pitfall: Chatty Models Break Letter-Only Prompts, Regex Grabs Wrong Answers — mancubus77 · 2026-11-05
- Full walkthrough: editing videos with Claude Code and Opus 5.5 — aakashgupta · 2026-11-05
- Claude Code can now edit your videos with a single command — aakashgupta · 2026-11-05