Selective Prefix Anti-Interference Regularization (SPAR) uses a short-context sufficiency gate and a gated KL objective to prevent language models from over-conditioning on irrelevant distant prefixes. This mechanism stabilizes suffix predictions when local context is already sufficient for the target token.
HOW THIS AFFECTS YOU
●
researcherYou can improve model robustness in long-context tasks by explicitly regularizing against irrelevant prefix perturbations.