New structured pruning method treats unit selection as a cardinality-constrained binary quadratic program to account for non-orthogonal weight correlations. A greedy interaction algorithm is used to solve the NP-hard optimization, reducing performance degradation compared to independent unit pruning.
HOW THIS AFFECTS YOU
●
builderYou can achieve more efficient LLM inference by utilizing pruning that accounts for activation correlations.
●
researcherThis method addresses the errors caused by the independence assumption in traditional pruning.