Rotation-Preserving Supervised Fine-Tuning

Rotation-Preserving Supervised Fine-Tuning (RPSFT) preserves singular subspaces of pretrained weights as efficient proxy for Fisher-sensitive directions, mitigating out-of-domain generalization degradation without expensive Hessian computation.