Privileged Information Does Not Guarantee Privileged Guidance: Auditing On-Policy Self-Distillation for Reasoning Models
November 13, 2026
Auditing of on-policy self-distillation across three Qwen3 scales reveals that providing a teacher model with worked solutions does not reliably improve student reasoning. Token targets from privileged teachers fail to favor correct trajectories, and repeated training improves performance only on tasks already within the student's reach rather than scaling reasoning capabilities.