[HUGGINGFACE]score: 0.48
PEFT-Arena Benchmark Shows Orthogonal Finetuning Best Balances Plasticity and Forgetting
May 26, 2026
PEFT-Arena jointly measures downstream task performance and general capability retention across PEFT methods, finding that orthogonal finetuning achieves the best stability-plasticity Pareto frontier under comparable parameter budgets. Spectral analysis of weight-space updates explains the geometric differences between methods.
paper
HOW THIS AFFECTS YOU
●
builderIf you are finetuning production models and need to preserve general capabilities, orthogonal finetuning is the empirically strongest choice under this benchmark.
●
researcherThe stability-plasticity framing and spectral analysis provide a principled basis for comparing PEFT methods beyond accuracy-only leaderboards.