Cradle Codec compresses KV cache for Ethernet transport between GPU nodes
knowrohit07 · x · 2026-07-29
The author describes Cradle Codec as a GPU-native KV-cache compression approach designed to move cached state over Ethernet instead of relying on NVLink.
Problem
- Reusable prefixes save compute, but KV cache state is large, expensive to keep around, and slow to move between nodes without fast interconnects.
Core idea
- Treat KV cache as structured temporal data, not a byte blob.
- Preserve token order as time.
- Keep attention heads structurally separated within each frame.
- Map adjacent layers into independent frame channels.
- Use dedicated codec engines for the transport format.
System implication
- Encoding can happen offline or outside the request path.
- Compressed artifacts can live on disk or behind a storage service.
- Decode/restoration can be pushed progressively closer to the serving engine.
Related event: Cradle Codec: GPU-Native KV Cache Compression for Cross-Node Migration(3 posts)→
More from Infra
- A GPU cloud business skeptics fear may actually be highly profitable — abhiadesai · 2026-07-29
- Andy Masley says critics are misreading his data-center arguments as prompt-cost math — AndyMasley · 2026-07-29
- A beginner-friendly video explains five GPU optimization methods for LLMs — 2C_ornot2C · 2026-07-29
- Jon Durbin says he pre-trained a 20B MoE for under $10 an hour — const_reborn · 2026-07-29
- Local LLM Setup: Is a Modded 3080 20GB Worth It Next to a 3090? — YourNightmar31 · 2026-07-29
- DIY local AI server uses retired NVIDIA cards for about $165 total — blelbach · 2026-07-29