RLVR Training Exhibits Primacy Bias Toward Easier Problems
November 4, 2026
Reinforcement Learning with Verifiable Reward (RLVR) suffers from primacy bias, disproportionately improving performance on problems the model already solves well. While hard problems provide the most significant reasoning expansion, RLVR tends to sharpen easy-task performance at the expense of exploring harder reasoning paths.
HOW THIS AFFECTS YOU
●
researcherYou should prioritize harder, novel problems in RLVR training datasets to avoid reinforcing existing model biases and to drive true reasoning expansion.