On-Policy Self-Distillation (OPSD) allows models to train on their own generations using privileged information like plans or feedback as a teacher. While reducing the need for larger teacher models, the method introduces significant signal bias due to the asymmetry between the student and the informed teacher.
HOW THIS AFFECTS YOU
●
researcherBe cautious when using OPSD for training, as the lack of an external teacher can introduce systematic biases into your model.