AbridgeKV Compresses Reasoning Chains into Learned Pause Tokens

AbridgeKV reduces the memory overhead of long chain-of-thought decoding by compressing text blocks into the KV entries of learned pause tokens. It optimizes the KV cache by allocating budgets across layers and allowing subsequent tokens to attend to these compressed representations.