Order-Only On-Policy Distillation Improves Student Model Training
October 31, 2026
O3PD improves on-policy distillation by training student models to match the ranked order of a teacher's top-K tokens rather than exact probability values. This makes the distillation process invariant to teacher temperature and prevents students from attempting to mimic uncalibrated teacher confidence.
HOW THIS AFFECTS YOU
●
builderYou can distill large models more effectively by focusing on token ranking rather than absolute probabilities.
●
researcherThis decouples preference information from teacher calibration, solving a key issue in distillation stability.