MIT's David Bau: have KV caches made RAM permanently expensive? Maybe takeoff is in $/bit
davidbau · x · 2026-09-23
MIT professor David Bau muses that computers have become expensive to buy and asks whether to wait for memory prices to drop—or whether KV caches have raised the real value of a bit of RAM so much that memory will never be cheap again. He notes in-context learning scales far better than anyone anticipated and jokes that "takeoff" might be measured in $/bit.
More from Infra
- Huawei unveils Peerium architecture: nested BSP unifies million processors into one computer — Dr_Singularity · 2026-09-23
- Grok explains why DeepSeek picked DualPipe + ZeRO-1 over ZeRO-3 on 2048 H800s — TheZachMueller · 2026-09-23
- AI costs fall 47% per quarter, 4x faster than DNA sequencing: Epoch AI — daveholtz · 2026-09-23
- M5 Ultra LLM test: 4x faster prompt processing, but double the power draw — DigitalguyCH · 2026-09-23
- $500 of Dell OptiPlexes become a diskless netboot lab where AI agents can't brick the hardware — colinmcnamara · 2026-09-23
- AMD MI355X beats NVIDIA B200 by 2.25x at scale with ~40% lower cost per GPU hour: Signal65 — ryanshrout · 2026-09-23