KV Packet treats cached documents as immutable packets wrapped in trainable soft-token adapters via self-supervised distillation. This method achieves near-zero FLOPs and lower Time-to-First-Token (TTFT) compared to recomputation-based methods like CacheBlend on Llama-3.1 and Qwen-3.
HOW THIS AFFECTS YOU
●
builderYou can reuse cached documents across different contexts without the latency penalty of recomputing KV states.
●
researcherThis provides a new architectural approach to mitigating context-dependency in KV caching.