Self-Preservation via Task Banks in Continual Learning

Using Task Banks and replay generation, language models can select specific targets to protect prior capabilities during continual instruction tuning without access to original training data.