KVpop: Predictive Online Pruning for KV Cache Compression
July 5, 2026
KVpop learns a fixed-budget eviction policy by supervising keep-or-drop decisions against a future-attention target. The method uses a delayed memory-based scorer to exploit near-future context, retaining 98% of utility on mathematical reasoning tasks.
HOW THIS AFFECTS YOU
●
builderYou can significantly reduce memory and bandwidth bottlenecks in long-context autoregressive decoding.