Lossless KV Cache Compression Up to 4x Using Predictor Models

Speculative KV coding uses a small predictor model to losslessly compress KV cache by up to 4x, reconstructing the cache exactly rather than absorbing quality loss from quantization. Unlike lossy approaches such as TurboQuant, this method lets you specify zero quality degradation upfront, which matters for long-context agentic workloads where cache storage and bandwidth dominate costs.