Bayesian Repetition Penalty: A Principled Adjacent-Conditional Framework for Reversing Attention Collapse in Autoregressive Language Models

Bayesian Repetition Penalty mitigates autoregressive attention collapse by penalizing token confidence based on the divergence between observed frequency and corpus priors. The framework utilizes a closed-form logit offset via an adjacent-conditional probability ratio, allowing for deployment as a non-intrusive repair mechanism through an exponential moving average of output-layer biases.