Teacher-Gated On-Policy Distillation Improves Student Training
September 4, 2026
Teacher-Gated On-Policy Distillation (TGOPD) prevents incorrect teacher signals from corrupting student models during post-training. By verifying teacher reliability at the prompt level using verifier-scored probes, the method routes prompts to dense distillation only when the teacher is reliable, otherwise using verifier-grounded GRPO.
HOW THIS AFFECTS YOU
●
builderYou can use prompt-level gating to prevent 'confidently wrong' teachers from inducing bad updates during distillation.
●
researcherThis approach mitigates the mode-seeking risks of reverse KL in vanilla on-policy distillation.