Mathematical proof of non-orthogonal interaction in model compression

New research proves that low-rank decomposition and quantization are not orthogonal when combined in language models. The findings show that combining these methods introduces complex error interactions that challenge the assumption of independent performance degradation.