Open-source TrnRT engine runs 320B GLM model on a single Trainium2

Frustrated by AWS Trainium's lack of support for recent open-source models, the RunAnywhereAI team open-sourced TrnRT, a NKI megakernel inference engine that serves a 320B MoE GLM model with 1M context at 3.57M tokens per minute on a single trn2.48xlarge instance.

2026-11-05 ~ 2026-11-05 · 2 related posts

• Open-source TrnRT engine serves GLM-5.3-Flash on one Trainium2 box at 3.57M tokens/min — ycombinator · 2026-11-05
• Open-source NKI megakernel engine serves 320B GLM-5.3-Flash on a single AWS Trainium2 box — WildCard727 · 2026-11-05