AI Infra Evolution: KVCache and SSDs Reshape the Inference Data Path
量子位 · wechat · 2026-08-07
As LLMs advance into long-context and multi-agent phases, AI infrastructure is shifting from merely 'piling up compute' to coordinating compute, network, memory, and storage. Moonshot's Mooncake and NVIDIA CMX exemplify this trend, treating inference states (like KVCache) as first-class resources integrated into the real-time token generation path.
The article notes that traditional SSDs cannot meet the strict timing constraints of LLMs, giving rise to 'AI SSDs'. The market is splitting into two core paths:
- AI-reinforced Enterprise SSDs: Maintain standard block storage forms but enhance bandwidth, endurance, and capacity (e.g., Innogrit, Huawei).
- Inference-participating AI SSDs: Deeply integrate storage into runtime management. Phison uses dedicated cache SSDs and middleware to extend VRAM; Longsys uses SPU and iSA for smart storage-side scheduling; Infplane-Maxio collaborates from the system level to schedule data precisely according to model execution logic.
The ultimate competition in AI SSDs will be a contest of complete data paths and software-hardware ecosystems.
More from Infra
- Amazon Proposes Per-Request Validated Evidence Caching for Production RAG — _reachsumit · 2026-10-16
- LinkedIn's TransX report: sequence transduction for recsys, +6.02% CTR in A/B tests — _reachsumit · 2026-10-16
- llama.cpp PR boosts CPU inference speed with merged AVX2 tiled accumulators — jacek2023 · 2026-10-16
- Zero's next-gen IVM engine runs in a single DO, live-querying a 100GiB R2 database — threepointone · 2026-10-16
- Meme: driving to the chip factory in 2028 captures AI's compute anxiety — beffjezos · 2026-10-16
- One volunteer maintains the time zone file behind 4 billion phones — danbri · 2026-10-16