SaMA PEFT Method Scales Rank Quadratically via Kronecker Products
November 4, 2026
SaMA improves upon LoRA by relaxing shared-block constraints in perfect-shuffle factorization, allowing effective rank to scale quadratically with the parameter budget. This approach also collapses the invariance manifold to a diagonal group, creating a more optimizer-friendly loss landscape.
HOW THIS AFFECTS YOU
●
builderYou can achieve higher fine-tuning performance with smaller parameter budgets using this method.
●
researcherThe quadratic rank scaling offers a more mathematically principled approach to parameter-efficient fine-tuning.