Yaroslav Bulatov launches A100-targeted architecture experiments attacking backprop's memory wall
yaroslavvb · x · 2026-09-12
Yaroslav Bulatov argues there's room for AI architectures customized to existing hardware and is experimenting on A100s. His open-source sutro-problems repo poses an MNIST challenge: hit target accuracy while addressing the memory wall — e.g., an A100 kernel measured in few Joules via NVML, or an algorithm with a small footprint on Bill Dally's 2D grid. Motivation: backprop dates from the arithmetic-bound 1980s; the bottleneck today is memory movement, and batching only partly mitigates it.
More from Infra
- Intel's silicon photonics couplers hit 1-1.5 dB IL, with visible epoxy delamination flaws — jwt0625 · 2026-09-12
- CPO paper criticized for vague DLW-to-PIC coupling description: 'such as TCB' — jwt0625 · 2026-09-12
- During AWS outage, one engineer kept enterprise services up with just 22 min downtime — generativist · 2026-09-12
- US hosts 43% of global datacenter power use; China just 13%, report finds — TMWNN · 2026-09-12
- Intel engineers show wafer-level chiplet testing for co-packaged optics paper — jwt0625 · 2026-09-12
- Intel engineers spotted testing chiplets and packages on wafer-level tester — jwt0625 · 2026-09-12