AI chip stacks are still wide open, with memory, networking, and optics left to improve
bookwormengr · x · 2026-07-25
AI chips are still far from settled
The post argues that Nvidia’s architecture is not necessarily the optimal one, and that there is plenty of room for new chip players to innovate at both the physical and logical layers. It cites Chinese AI chip companies as mostly focused on design/logical-layer innovation, with a few exceptions moving toward wafer-scale approaches.
It also points to where the real room for improvement still is: memory access, networking, RAM pooling, and optical interconnects that can move closer to the chip. The underlying claim is that modern LLM stacks are path-dependent on the hardware they were built on, so better architectures may still be waiting to be discovered.
More from Infra
- Encrypted GPT-2 speedrun challenges FHE inference: best is 3.031 s/token, goal sub-1s — tensorqt · 2026-10-27
- Open-source Latch compiles PyTorch into training megakernels, 2.75x faster on H100 — sguada · 2026-10-27
- Stanford launches GPU-centric supercomputer Marlowe for data science and AI research — StanfordHAI · 2026-10-27
- NVIDIA Claims HGX Rubin NVL8 Delivers 10x More Tokens per Megawatt Than B200 — thursdai_pod · 2026-10-27
- Together lets multiple LoRA post-training experiments share the same GPU capacity — togethercompute · 2026-10-27
- Commoditization dilemma: either human labor or GPUs become the commodity — _AustinCalvert_ · 2026-10-27