Reward-Aligned On-Policy Distillation to Prevent Misleading Teacher Guidance
August 31, 2026
RA-OPD improves on-policy distillation by filtering teacher model guidance that contradicts outcome rewards. The method identifies and retains only trajectories where teacher-induced updates align with the intended outcome reward, preventing performance degradation during student training.
HOW THIS AFFECTS YOU
●
builderYou can implement this to ensure student models trained via distillation do not inherit misaligned teacher behaviors.
●
researcherThis method provides a way to improve the stability of knowledge transfer in distillation processes.