vLLM creators' Inferact now serves 7 trillion tokens a day
woosuk_k · x · 2026-11-06
Inferact, founded by vLLM's creators and core maintainers, now serves over 7 trillion tokens daily across heterogeneous hardware (NVIDIA, AMD, Google accelerators), for context OpenRouter averaged 19T daily open-weight tokens. The platform acts as a control plane where agents tune each model-chip-workload deployment, handles agentic workloads with dynamic resource needs, and runs a data flywheel: production traffic improves vLLM, which feeds back into the platform, with agents next automating optimization discovery.
Related event: Inferact, Founded by vLLM Creators, Serves Over 7 Trillion Tokens Daily(2 posts)→
More from Infra
- Musk says Terafab 'moving fast' as Tesla's chip fab goes up in days — elonmusk · 2026-11-06
- SoftBank picks Malaysia for $50B tech investment in global AI bet — basedjensen · 2026-11-06
- Why no high-quant coding-pruned DeepSeek or GLM for 256GB machines? — evrenozkan · 2026-11-06
- Data center capex reportedly fleeing en masse to Malaysia — basedjensen · 2026-11-06
- Homelab with 3 mixed-GPU machines: how do you track local LLM models and VRAM? — pipecanon · 2026-11-06
- ING said to be seller of US$500M data centre loan portfolio — Bloomberg — apples_jimmy · 2026-11-06