Wafer launches 'most comprehensive' AI performance engineering repo, starting with Transformer inference deep-dive
ycombinator · x · 2026-09-12
Wafer last week released what it calls the world's most comprehensive AI performance engineering repo and is now publishing every resource from the series. Part 1, "All About Transformer Inference" from How To Scale Your Model, covers:
- Arithmetic intensity of linear layers and attention across prefill and decode
- KV cache sizing by layer count, KV heads, head dimension, sequence length, and precision
- The compute/HBM bandwidth crossover and how batch size and quantization shift it
- Decode latency and throughput bounds from parameter bytes, KV bytes, and hardware bandwidth
- Weight reuse via batching and diminishing throughput gains as KV traffic grows
- GQA, KV quantization, and PagedAttention
The author suggests saving the thread as a starting point, with links in the thread.
Related event: Wafer Launches Comprehensive AI Performance Engineering Resource Library(2 posts)→
More from Infra
- Anthropic in talks to lease 920MW gas-powered data centre campus near Edmonton — rohanpaul_ai · 2026-10-31
- GPU price index tracks 114 rates from 25 vendors, free MCP server for agents — StasBekman · 2026-10-31
- Intel stock crash wipes $12B off US government's stake as AI eats the PC supply chain — TechCrunch AI · 2026-10-31
- CMU paper unifies pretraining, SFT and RL into one likelihood-based training algorithm — rohanpaul_ai · 2026-10-31
- Boom Supersonic airliner in doubt as company pivots to AI data center turbines — Ars Technica AI · 2026-10-31
- AWS adds AgentCore Session endpoints to preview coding agent web apps live in browser — AWS ML Blog · 2026-10-31