Wafer launches 'most comprehensive' AI performance engineering repo, starting with Transformer inference deep-dive
ycombinator · x · 2026-09-12
Wafer last week released what it calls the world's most comprehensive AI performance engineering repo and is now publishing every resource from the series. Part 1, "All About Transformer Inference" from How To Scale Your Model, covers:
- Arithmetic intensity of linear layers and attention across prefill and decode
- KV cache sizing by layer count, KV heads, head dimension, sequence length, and precision
- The compute/HBM bandwidth crossover and how batch size and quantization shift it
- Decode latency and throughput bounds from parameter bytes, KV bytes, and hardware bandwidth
- Weight reuse via batching and diminishing throughput gains as KV traffic grows
- GQA, KV quantization, and PagedAttention
The author suggests saving the thread as a starting point, with links in the thread.
More from Infra
- Zilliz CTO: Agent memory is a long-lived systems problem, not an index feature — J_Luan_ · 2026-09-12
- Draw Things update adds MiniMax H3 with LoRA/TeaCache and Krea 2 model imports — antirez · 2026-09-12
- Together serves 23%-30% of all OpenRouter traffic for GLM 5.3 models — zhyncs42 · 2026-09-12
- The Global Race for Cheap Power: Where AI Data Centers Should Actually Go — pravchaw · 2026-09-12
- VCs float 'hardware revenue derivative': fund compute costs via revenue share, not equity — ns123abc · 2026-09-12
- Relace hits 1T tokens/day on OpenRouter, serving 37% of DeepSeek v4 Flash traffic — stuffyokodraws · 2026-09-12