Teacher-Gated On-Policy Distillation (TGOPD) adds prompt-level verification to the distillation process. By estimating teacher reliability before applying dense supervision, it prevents mode-seeking updates from confidently wrong teachers that otherwise induce misleading student behavior.
HOW THIS AFFECTS YOU
●
builderYou can improve distillation stability by gating supervision based on teacher correctness.
●
researcherThis addresses the risks of reverse KL in vanilla on-policy distillation.