Latent-LoRA: Gradient-Free Routing for Task-Agnostic Continual Learning
July 28, 2026
Latent-LoRA enables continual learning in LLMs by using a Gaussian mixture model on frozen embedding layers to select LoRA adapters. This gradient-free routing eliminates the need for trainable gating modules that are prone to catastrophic forgetting.
HOW THIS AFFECTS YOU
●
builderYou can implement more efficient continual learning pipelines without the overhead of training task-specific routers.
●
researcherThis provides a mechanism for task-agnostic adapter selection using existing embedding distributions.