INT21 claims 20 AI-generated inference engines in 2 weeks, MiMo hits 1,308 tok/s
bingxu_ · x · 2026-09-29
Bing Xu shared INT21's inference benchmark, arguing human-crafted AI software is becoming history. INT21 positions ultra-fast inference as the bottleneck of the agentic era: a coding agent makes dozens of sequential calls per fix. Two engineers directed generation of 20 engines in 2 weeks across 7 model categories (autoregressive/diffusion LM, ASR, TTS, OCR, image, video). On a 15-model subset vs SGLang and vLLM, INT21 leads decode rate on all 6 text models tested: MiMo at 1,308 tok/s vs 540 for tuned SGLang and 1,011 for vLLM.
More from coding & agent
- Google Research open-sources PaperOrchestra, a multi-agent framework for writing AI papers — google-research · 2026-11-16
- ffmpeg-skill gives coding agents local video editing via 42 FFmpeg tools and MCP — kajisho5 · 2026-11-16
- Agents Now Reply to Slack Messages on Their Own, Resolving a Prod Incident Ghost-to-Ghost — iamrobotbear · 2026-11-16
- Engineer's Interview Prep Method: Use Claude to Find Which Answers Undersell Your AI Work — brandon_galang · 2026-11-16
- DeepSeek open-sources its app frontend "chat-demo", already at 3.9k stars — NielsRogge · 2026-11-16
- lemire: AI's real impact on coding is expanding the engineer's 'cognitive radius', not speed — lemire · 2026-11-16