A Step Harder, Not a Step Too Far: The Benefits and Limits of Curriculum Difficulty Progression in GRPO

Title: A Step Harder, Not a Step Too Far: The Benefits and Limits of Curriculum Difficulty Progression in GRPO Source: arxiv Staged curriculum learning improves GRPO training efficiency by progressively shifting the active difficulty band toward harder samples rather than persistently selecting medium-difficulty ones. On the GSM8K benchmark, re-estimating sample difficulty relative to the evolving policy prevents misalignment and achieves higher final model performance compared to static difficulty selection.