The study identifies that on-policy distillation (OPD) success depends on 'where to branch' in reasoning paths rather than just 'how to branch' via token overlap. Introducing Entropy Shape Correlation (ESC) allows for measuring student-teacher agreement in next-token entropy sequences to predict distillation performance.
HOW THIS AFFECTS YOU
●
builderYou can better predict whether a stronger teacher model will actually improve your student model's reasoning.
●
researcherThis offers a new metric, ESC, to diagnose and improve post-training distillation workflows.