Functional Subnetwork Localization for Efficient Continual Learning in Large Language Models

CiMEO uses gradient-attribution profiling to localize task-specific functional subnetworks within LLMs, minimizing parameter overlap between tasks. By routing training examples to these specific regions, the framework restricts updates and optimizer states to selected subnetworks. It maintains superior average performance across five generative benchmarks during 1.4 million sequential instruction tasks.