RSTG Mitigates Sparse Rewards in GRPO via Adaptive Teacher Distillation
August 4, 2026
RSTG addresses gradient loss in Group Relative Policy Optimization by selectively applying on-policy distillation only when group rewards are identical. This adaptive guidance prevents the exploration degradation typically caused by naive distillation, providing denser token-level signals without undermining the RL agent's capacity.
HOW THIS AFFECTS YOU
●
builderThis offers a method to improve LLM post-training efficiency when using reward models with low variance.
●
researcherYou can leverage adaptive distillation to solve gradient sparsity in RLVR frameworks.