MM-ShiftKV is a training-free, decode-aware KV selection method designed to stabilize multimodal large language model inference. It addresses the high variance in visual token importance during decoding, preventing the disproportionate loss of critical visual information caused by prefill-only selection methods.
HOW THIS AFFECTS YOU
●
builderYou can reduce the memory footprint of multimodal models without sacrificing grounding performance by using decode-aware selection.