Cradle Codec compresses KV cache for Ethernet transport between GPU nodes
knowrohit07 · x · 2026-07-29
The author describes Cradle Codec as a GPU-native KV-cache compression approach designed to move cached state over Ethernet instead of relying on NVLink.
Problem
- Reusable prefixes save compute, but KV cache state is large, expensive to keep around, and slow to move between nodes without fast interconnects.
Core idea
- Treat KV cache as structured temporal data, not a byte blob.
- Preserve token order as time.
- Keep attention heads structurally separated within each frame.
- Map adjacent layers into independent frame channels.
- Use dedicated codec engines for the transport format.
System implication
- Encoding can happen offline or outside the request path.
- Compressed artifacts can live on disk or behind a storage service.
- Decode/restoration can be pushed progressively closer to the serving engine.
Related event: Cradle Codec: GPU-native KV Cache Compression for Ethernet Transfer(4 posts)→
More from Infra
- Meta's commitments jumped to $349B in a quarter as server memory prices tripled ahead of Oct 21 earnings — tengyanAI · 2026-10-16
- Meta releases Muse Spark Open, trained with vLLM-powered RL inference from day one — colinmcnamara · 2026-10-16
- Meta's open Muse Spark needs 493GB of memory — far beyond any consumer laptop — TejasKumar_ · 2026-10-16
- CoreWeave plunges 20% as credit markets price >60% default odds; is this dot-com all over again? — kimmonismus · 2026-10-16
- MiniCPM-V 4.7 35B-A3B hits 17+ tokens/s on iPad with ~3GB memory — techNmak · 2026-10-16
- China rolls out 'token factories' and a 2026-28 token economy action plan — nordicinst · 2026-10-16