SLORR is a stateless, architecture-preserving regularization framework designed to improve model compressibility during training. It uses GPU-friendly approximations of the Hoyer sparsity metric and nuclear norm to regularize weight matrices without requiring SVDs or additional trainable parameters.
HOW THIS AFFECTS YOU
●
builderYou can reduce model footprint and deployment costs by applying low-rank regularization during fine-tuning.
●
researcherYou can improve the compressibility of models during continued training without the overhead of SVD-based methods.