vLLM grew from a two-week side project to 500K GPUs; team raised $150M
a16z · x · 2026-10-16
a16z profiles vLLM's origin: in 2022, Berkeley PhD student Woosuk Kwon set out to speed up Meta's open-weight OPT-175B in two weeks. Four years and 3,000 contributors later, it's one of the largest open-source inference engines, running on 500,000 GPUs at any given time.
The key idea borrowed "virtual memory" from operating systems, allocating GPU memory on demand so each card juggles requests of wildly different sizes. Its first user, LMSYS's Chatbot Arena, cut GPU needs in half; today Google, Amazon, and Meta all run vLLM in production.
In January, core maintainers founded Inferact and raised $150M to staff vLLM full-time, now hiring eng, marketing, and devrel in SF and Singapore.
More from Companies & People
- Mintlify announces partnership with Xbox team to power its content — handotdev · 2026-10-17
- Google engineer teases Beam tech report arriving later this month — alexpolozov · 2026-10-17
- Fireworks, Microsoft and NVIDIA to host SF buildathon with $10K top prize for beating frontier models — sophiamyang · 2026-10-17
- Margaret Hamilton, Apollo software pioneer who coined 'software engineering,' dies at 90 — Dr_Atoosa · 2026-10-17
- Countering a16z's 'Europe left behind' claim: EU firms lead multiple AI categories — IsForAt · 2026-10-17
- Datology Labs opens paid 6-month data research fellowship, apply by Nov 15 — josh_wills · 2026-10-17