RSPO Framework for Regularized Self-Play Alignment

RSPO is a new framework for self-play alignment that unifies various regularization strategies to mitigate over-optimization. It demonstrates improved length-controlled win rates on AlpacaEval-2 and superior performance on Arena-Hard and MT-Bench across multiple base models.