Mitigating Repetition Loops in Pruned LLMs via FOCUS and RePAIR
August 28, 2026
Pruning causes text degeneration and repetition loops by reducing the escape mass available during token sampling. The proposed FOCUS and RePAIR objectives use token-level guidance during post-pruning fine-tuning to suppress loop entry and promote plausible alternatives.
HOW THIS AFFECTS YOU
●
builderYou can use these methods to maintain high-quality text generation when deploying pruned, low-latency models.
●
researcherYou can use these specific loss functions to stabilize the dynamics of decoding in compressed models.