KV-Repurpose Enables Segment-Wise KV-Cache Reuse in vLLM
October 12, 2026
KV-Repurpose allows for non-prefix KV-cache reuse by enabling segment-wise retrieval of cached data across different prompts or histories. In Zipfian RAG workloads, it increased the token-level cache hit rate from 3% using standard prefix caching to 63.9% at 30 GB capacity.
HOW THIS AFFECTS YOU
●
builderYou can significantly reduce inference latency and costs for RAG systems by reusing document segments regardless of prompt position.
●
founderThis improves the unit economics of serving long-context models in production.