4 essential reads to understand LLM decode from first principles

CatAstro_Piyush · x · 2026-11-21

A curated list of 4 classic reads for understanding LLM decode from first principles: kipply's Transformer Inference Arithmetic (KV cache, capacity, tensor parallelism, batch sizes, FLOPS counting, benchmark comparisons), Flash-Decoding by Tri Dao et al., PyTorch's gpt-fast writeup, and the original speculative decoding paper. Ideal for engineers who want to reason about inference latency, memory bandwidth, and parallelism costs arithmetically.

Original post →

More from Infra

Infra channel →