From a Prompt to Repertoires: Evolving Functional REpertoires Enable LLM Continual Learning

Evolving Functional REpertoires (EFRE) addresses catastrophic forgetting in prompt-based continual learning by replacing static prompts with an evolving set of task-specific rules. The method prevents overfitting to local task distributions by managing a repertoire of functional instructions, offering an efficient alternative to parameter-heavy updates like GRPO.