Rethinking KV Cache Eviction via a Unified Information-Theoretic Objective

CapKV (arXiv:2504.25975) reframes KV cache eviction as an Information Bottleneck optimization problem, deriving a closed-form mutual information objective under a linear-Gaussian attention surrogate. The method uses log-determinant approximation via statistical leverage scores, unifying existing heuristics like H2O and SnapKV as special cases of one capacity-maximization principle. Engineers targeting long-context inference on memory-constrained hardware should evaluate CapKV immediately, as it offers theoretically grounded token eviction replacing ad-hoc attention-score thresholds with no architectural changes required.