Full talk slides released: how inference engines actually work, end to end
zainhas · x · 2026-09-12
zainhas gave a talk titled "how inference engines actually work" at the Presidio Bitcoin Open Source AI summit and released the full slide deck, covering the end-to-end lifetime of a request:
- the inference engine itself
- KV and prefix caching
- continuous batching
- Paged Attention
- chunked prefill
- sampling
- agentic loops from inside the engine
A solid resource for developers who want a systematic understanding of the LLM inference serving stack.
Related event: Full Slide Deck Explaining How Inference Engines Actually Work Goes Public(4 posts)→
More from Infra
- Cornell Professor's Startup Makora Wins $1.75M to Make AI Faster and Cheaper to Run — srush_nlp · 2026-10-31
- Transformers.js monthly npm downloads grow 9x in a year, from 1.64M to ~15M — nicodotdev · 2026-10-31
- Agents per user doubling every 6 months: AgentWrap rearchitects runtime around 'agent as a function' — matanSF · 2026-10-31
- Anthropic's personnel costs surged 486% to $3.15B, with stock comp at 70% — rohanpaul_ai · 2026-10-31
- Dual 3090 + 32GB RAM can't run Qwen Strata low-bit quants; user mulls a $479 RAM upgrade — runcertain · 2026-10-31
- a16z: B300 GPU rentals up 106% since July while average token prices fall 58% — a16z · 2026-10-31