SigmaScale Learns Scaling Matrices to Improve SVD-Based LLM Compression on Llama and Qwen3

SigmaScale optimizes diagonal row and column scaling vectors under an activation-aware loss before SVD truncation, reducing effective intrinsic rank of weight matrices and improving compression quality. Benchmarked on Llama 3.1 8B Instruct and Qwen3-8B, it matches state-of-the-art SVD compression methods on perplexity and zero-shot tasks.