RSHD Recovers Supervision from Failed GRPO Training Rollouts
October 16, 2026
Replay-Selected Hindsight Distillation (RSHD) allows GRPO to learn from failed reasoning attempts by having the policy write corrective notes. These notes are only distilled if they facilitate successful subsequent replays, enabling supervision even when all initial group attempts fail.
HOW THIS AFFECTS YOU
●
researcherYou can mitigate the signal loss in RL training when model rollouts consistently fail to reach a target state.