S2D-OPD Improves Distillation via Selective Token Supervision
September 25, 2026
Selective Supervision for Direct-OPD (S2D-OPD) addresses inefficiencies in policy distillation where log-ratio rewards fail to capture vanishing probability mass. The method ranks student-sampled states by teacher-reference Jensen-Shannon divergence to prioritize tokens that provide meaningful policy improvements.
HOW THIS AFFECTS YOU
●
builderThis provides a more effective way to distill large-scale model capabilities into smaller, production-ready weights.
●
researcherYou can use this masking strategy to improve the efficiency of transferring RL improvements to smaller models.