Open-source LLM Optimization Handbook: 48 topics with hands-on quantization, KV cache and kernel problems
udmrzn · x · 2026-11-18
TensorTonic open-sourced the LLM Optimization Handbook, a roadmap for making LLMs faster, cheaper and smaller. It covers 48 topics across 7 stages — inference fundamentals, quantization, KV cache, speculative decoding, GPU kernels and serving — each with a short explanation and a hands-on problem where you implement the idea in Python, CUDA or Triton and check against tests. The repo also curates the 49 most common inference/LLM engineer interview questions with explanations and solvable problems. MIT licensed.
More from Infra
- Free online book: LLM Inference Handbook for production deployment — burkov · 2026-11-18
- SEC opens civil probe into Nvidia over undisclosed China-linked chip sales — The Decoder · 2026-11-18
- Nvidia stock barely moves after blowout quarter, investor holds off buying the day-one pop — bindureddy · 2026-11-18
- One HF Job, $127 total: Inkling-2-Small image generation under a tenth of a cent per picture — vanstriendaniel · 2026-11-18
- Voice agent team cuts P95 time-to-first-voice by 50% via single-datacenter migration — itsOmSarraf_ · 2026-11-18
- Satnam meets Prof. George Constantinides in Cambridge on UK AI infrastructure — satnam6502 · 2026-11-18