LHM Framework Uses Second-Order Curvature for Efficient Model Merging

Low-Rank Hierarchical Merging (LHM) enables training-free merging of long chain-of-thought models with concise base models. It utilizes a low-rank Hessian approximation and a Layer Second-Order Sensitivity index to avoid the accuracy degradation typical of linear interpolation methods.