Adaptive On-policy Self-distillation Improves LLM Post-training

Multi-teacher adaptive on-policy self-distillation (OPSD) selects a specific teacher for every token based on KL-based selection and majority voting. This approach addresses token-level teacher preference variance that fixed-teacher distillation methods overlook.