Open Field Guide to Inference Engineering: KV Cache math to vLLM/SGLang production checklist
udmrzn · x · 2026-11-12
An open tutorial (Modelplane v0.7) for platform engineers new to LLM inference, covering GPU fundamentals, KV Cache memory estimation, Prefill/Decode bottlenecks, vLLM and SGLang, inference optimization, production monitoring, Eval, plus a full go-live checklist. Ends with curated further reading from the inference community.
More from Infra
- Musk reportedly betting on Terafab: one plant producing a terawatt of silicon chips per year — XFreeze · 2026-11-12
- Physics-informed learning by IITM professor slashes AI simulation compute costs — ravi_iitm · 2026-11-12
- Precision on Demand: Escalating Weight Bits at Uncertain Decoding Steps — Mohammed Nayeem Khan · 2026-11-12
- MiniMax-H3-Image FP8 fits a single 32GB AMD card: 4K images in 33s — evp-cloud · 2026-11-12
- Energy Tech Draws Nearly $200B in 2025 as Data-Center Loads Test Grid Interconnection Queues — ingliguori · 2026-11-12
- vLLM-Omni brings Day-0 support for MiniMax H3-Image: native 4K in 0.87s — vllm_project · 2026-11-12