GVPO++ Stabilizes LLM Post-Training via Group Variance Policy Optimization

GVPO improves upon GRPO by integrating an analytical solution for KL-constrained reward maximization into its gradient weighting. This eliminates the training instability caused by importance sampling and guarantees a unique optimal solution for the reward objective.