Random Attention Increases KV Cache Throughput by 32-43%
September 2, 2026
Random Attention improves reasoning model efficiency by evicting KV cache tokens uniformly at random within each attention head rather than using expensive scoring mechanisms. In vLLM deployments, this method matches top-scoring evictors while providing 32-43% higher throughput.
HOW THIS AFFECTS YOU
●
builderYou can achieve significantly higher inference throughput for long-context reasoning tasks with minimal accuracy loss.
●
researcherThe finding suggests selection signals for KV cache eviction may be less critical than previously assumed.