Analyzing the impact of on-policy distillation on reasoning models

This research investigates how on-policy distillation affects the training of reasoning models by providing per-token supervision. The study identifies specific conditions where the distillation signal is either beneficial or detrimental to model performance.