J.B. Huang's new video explains how LLM serving actually works, from fast to cheap
jbhuang0604 · x · 2026-11-06
AI researcher J.B. Huang (jbhuang0604) released a new video, "How LLM Serving Actually Works," breaking down the key techniques that make LLM inference faster, cheaper, and more scalable — a broad tour of what happens behind the scenes when you call an LLM API.
More from Infra
- Blogger Proposes Data Centers Win Locals Over With Free ChatGPT Pro and Waste Heat — flowersslop · 2026-11-06
- Voters reject data centers in 40 of 41 ballot measures; Musk eyes orbital data centers — PeterDiamandis · 2026-11-06
- Ripple-65B open-weight model runs at 7 tok/s locally on a 64GB Mac mini — DZhang50 · 2026-11-06
- Liquid AI: a 9B LLM can't run in a car, but LFMs can — JosephJacks_ · 2026-11-06
- Nvidia chips pile up in Hong Kong as China withholds import approval — pstAsiatech · 2026-11-06
- Chart: UK IT investment at 0.66% of GDP vs US 2.21%, missing the AI capex boom — KeeganMcB · 2026-11-06