LoRA Superposition Enables Task Transfer Across Different Backbones
October 23, 2026
LoRA superposition demonstrates that Low-Rank Adaptation weights trained on different backbones can be summed and applied to any of those backbones. In tests across seven ViT-B/16 CLIP backbones, the summed LoRA achieved a mean accuracy within 0.28 percentage points of individually trained models.
HOW THIS AFFECTS YOU
●
builderYou can potentially aggregate fine-tuned weights from multiple models to create a single, versatile adapter.
●
researcherYou can investigate the orthogonality of LoRA parameter spaces to develop more modular fine-tuning techniques.