Study: Random KV Cache Eviction Beats Scoring Methods
A Salesforce and UIUC paper shows that scoring-based KV cache eviction adds little value; randomly evicting reasoning tokens can achieve 32-43% higher throughput.
2026-09-04 ~ 2026-09-04 · 2 related posts
- KV Cache Scoring Is Useless: Random Eviction Matches Top Evictors at 32-43% Higher Throughput — burny_tech · 2026-09-04
- Salesforce research: Random eviction of reasoning tokens matches selective KV cache compression — Salesforce · 2026-09-04