Inferact, founded by vLLM creators, now serves 7 trillion tokens a day
Xianbao_QIAN · x · 2026-11-06
Inferact, founded by the creators and core maintainers of vLLM, now serves over 7 trillion tokens daily across heterogeneous hardware and agentic workloads. For context, OpenRouter averaged 19 trillion tokens/day of open-weight traffic over the past 30 days. The company says serving at this scale means small inefficiencies compound across thousands of GPUs, hardware faults happen daily, and agentic workloads with long context prefixes and tool-call pauses expose bottlenecks static benchmarks miss. Inferact Platform acts as a control plane, with agents tuning each deployment for its model/chip/traffic mix to boost throughput at steady latency, while a data feedback loop between its fleet and vLLM development improves both.
More from Infra
- When neighborhoods vote you off the planet, build AI data centers in orbit — r0ck3t23 · 2026-11-06
- With US data centers zoned out, builders eye orbit where no one can vote you off the planet — r0ck3t23 · 2026-11-06
- Meta looks abroad for data centers as AI firms scramble for compute — asusarla · 2026-11-06
- CoreWeave brings 22.5MW Copenhagen site online, builds first global AI backbone — thursdai_pod · 2026-11-06
- Open-source inferd runs 128 concurrent agents on one Mac at 1,487 tok/s — thursdai_pod · 2026-11-06
- Voters restricted data centers in 38 of 39 local ballot measures — thursdai_pod · 2026-11-06