Deep learning lore: BatchNorm (2015) influenced early TPU v2 design; architecture research boom expected
gordic_aleksa · x · 2026-09-12
Aleksa shares a bit of deep learning history: batch normalization appeared in 2015, took the field by storm, and even influenced the design of early TPU v2 hardware — crediting Christian Szegedy.
He predicts an explosion of architecture research over the next few years, if not months.
More from Infra
- DeepSeek v4.1 Flash runs out of the box on six NVIDIA GPUs via vLLM on day 0, AMD lags — woosuk_k · 2026-09-12
- Open-source Plano routes LLM calls by prompt intent, no agent code changes, cutting bills 2x — Roger_M_Taylor · 2026-09-12
- Netflix Engineer Open-Sources Headroom, Cuts Agent Token Use by Up to 95% — Roger_M_Taylor · 2026-09-12
- DeepSeek V4.1 Flash cuts global KV cache to 890 bytes/token, but HBM demand may rise with agent swarms — teortaxesTex · 2026-09-12
- Polymarket Prices AI Bubble Burst at 12% Odds Through End of 2026 — Polymarket · 2026-09-12
- SGLang hits 873 tok/s on DeepSeek V4.1 Flash within 24 hours of launch — BanghuaZ · 2026-09-12