InertiaKV Increases KV Cache Throughput via EMA Aggregation
September 4, 2026
InertiaKV uses exponential-moving-average (EMA) aggregation for aggressive decoding-time KV cache eviction, achieving 1.34-1.46x decode throughput. The method preserves ranking and stability compared to traditional token scoring functions that fail under aggressive compression.
HOW THIS AFFECTS YOU
●
builderYou can achieve significantly higher throughput in long-context inference by using EMA-based eviction methods.
●
researcherTemporal aggregation rules are as critical as the scoring functions themselves for effective KV cache compression.