YC Paper Club: inference matters 12x more than training, worth $220B in market cap
Y Combinator · youtube · 2026-10-24
This week's YC Paper Club digs into inference: for every FLOP spent training a model, roughly six go to serving it each year — and getting inference right has produced over $220 billion in market cap over the past decade.
- Full lifecycle of an inference request, from API call down to the GPU kernel (Dan Fu)
- The gap between the best and worst providers serving the same open model is bigger than the gap between open and closed models (Jeff Morgan)
- A compiler built on search rather than heuristics turns GPU hours into tokens (Joe Fioti)
- How SGLang manages memory for the agent era (Lianmin Zheng)
More from Infra
- Physical AI Weekly: force-sensing gloves, Isaac Lab 3.0, humanoids in Audi plants — AIFlow_ML · 2026-10-24
- Chart: Hyperscalers' combined free cash flow projected to collapse near zero by 2026 — SumitGup · 2026-10-24
- Unlocking hidden RTX 4070 Ti SUPER compute: FP16 matmul gets 1.9x faster — lxfater · 2026-10-24
- a16z charts: US electrical equipment capacity down 9% since 2019 as prices jump 69% — a16z · 2026-10-24
- Flux.3 Klein generates 768x1024 images in 15s on an M5 iPad Pro with a 4-step Distill LoRA — JosephJacks_ · 2026-10-24
- Open-source framework streams any MoE model on any device: 35B at 25 tok/s on a 16GB Mac — dai_app · 2026-10-24