Random Attention Increases KV Cache Throughput by up to 43%
September 4, 2026
Random Attention improves reasoning efficiency by evicting KV cache tokens uniformly at random within each attention head instead of using scoring signals. In vLLM deployments, this method matches top-tier evictors while delivering 32-43% higher throughput across four models.
HOW THIS AFFECTS YOU
●
builderYou can achieve significantly higher inference throughput for long-context reasoning tasks by simplifying cache eviction.
●
researcherThis challenges the necessity of selection signals in KV cache management for reasoning-heavy workloads.