Randomized Design Enables Error Certificates for KV-Cache Eviction
July 24, 2026
Deterministic KV-cache eviction prevents consistent error estimation, but randomized Poisson-sampled eviction restores identifiability. This method provides a per-step error certificate with 0.97 empirical coverage and allows question-aware eviction at 25-50% budgets with minimal accuracy loss.
HOW THIS AFFECTS YOU
●
builderYou can implement more aggressive KV-cache eviction strategies with reliable error monitoring.
●
researcherThe proof shows deterministic eviction is fundamentally unidentifiable for error estimation.