Tsinghua's Cache Engineering Lets Agents Control KV Caches Without Re-prefilling
nics-efc · hf · 2026-10-22
Researchers from Tsinghua and CUHK propose cache engineering, letting LLM agents directly manage active KV caches via drop and repos operations without modifying trajectories or re-prefilling retained context. The paper formalizes history dependence—identical token sequences yield different cache states—invalidating prefix matching, and co-designs a serving engine.
More from Infra
- Sundar Pichai: Google Cloud could be selling quantum deployments at scale within 5 years — rohanpaul_ai · 2026-10-22
- TSMC surveys suppliers on rare-earth-free motors for factory automation — pstAsiatech · 2026-10-22
- Zoho cries poor over AI costs, then applies to build $200M semiconductor fab — kingslyj · 2026-10-22
- NVIDIA recaps GTC Berlin dev community: open models to physical AI — NVIDIAAI · 2026-10-22
- GPU payback math: B300 rigs recoup costs in 10 months, 1GW build could earn ~$38B a year — tctjr · 2026-10-22
- What do customers actually get for billions in AI capacity prepayments? 20 companies reviewed — tengyanAI · 2026-10-22