KV-Cache Patching identifies drivers of fine-tuned model behaviors

By patching KV caches between LoRA fine-tuned and base models, researchers found that fine-tuned behaviors like German language proficiency or misalignment can be transferred or removed. The effect is concentrated at pivotal positions such as system prompts and chat templates.