PaLoRA: Paced Low-Rank Adaptation for Continual Learning

PaLoRA addresses catastrophic forgetting in LoRA-based continual learning by replacing fixed, small learning rates with an adaptive magnitude restriction. The method scales update constraints based on the effective rank of historical knowledge to mitigate subspace leakage caused by finite-precision updates.