O'Reilly Releases Hands-On LLM Serving and Optimization, a Guide to Hosting LLMs at Scale
kalyan_kpl · x · 2026-11-03
O'Reilly's new book Hands-On LLM Serving and Optimization: Hosting LLMs at Scale by Chi Wang & Peiheng Hu is a comprehensive guide to deploying and optimizing LLMs at scale.
- Chapter 1 covers model serving fundamentals: model anatomy, lifecycle, serving paradigms (edge, single-model, multi-model), and platforms
- Chapter 2 dives into transformer internals and autoregressive generation serving
A systematic reference for engineers looking to cut inference cost and latency.
More from Infra
- Anthropic open-sources FlashProve: provable inference costs ~165x more than raw inference — arthurcolle · 2026-11-03
- PostHog re-rebuilds its data warehouse five months after switching from ClickHouse to DuckDB — teodorio · 2026-11-03
- Local MoE inference on a Tesla V100: 55 tok/s decode with 8,109 experts cached in 31.6GB VRAM — Traditional_Bell8153 · 2026-11-03
- Cities Worldwide Build AI Services on NVIDIA Nemotron, Cosmos and Omniverse — NVIDIAAI · 2026-11-03
- Anthropic spends $2.4B ARR on data vendors — just 6% of its compute bill — himanshustwts · 2026-11-03
- StoneX sees $309B in free cash flow losses for hyperscalers and neoclouds next year — GaryMarcus · 2026-11-03