Two-Channel Constrained GRPO Fixes Safety Budget Failures in VLMs
November 13, 2026
Standard GRPO fails to enforce safety budgets in vision-language models because scorers built for ranking have mean bias and within-group standardization cancels out penalty multipliers. The proposed two-channel constrained GRPO preserves raw scores for comparison to ensure multipliers correctly regulate harmful assistance and over-refusal rates.
HOW THIS AFFECTS YOU
●
builderYou can better control how often your models refuse safe requests or allow harmful ones.
●
researcherYou can use this method to prevent safety multipliers from being neutralized during reinforcement learning.