STOPD accelerates on-policy distillation via stochastic truncation
October 16, 2026
Stochastic Truncated On-Policy Distillation (STOPD) reduces the computational overhead of teacher-student training by randomly truncating student rollouts. It uses blockwise inverse-probability weighting (IPW) to correct sampling bias and preserve the expected gradient sum.
HOW THIS AFFECTS YOU
●
researcherYou can significantly speed up on-policy distillation training loops without losing gradient integrity.